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Preface 


During the past fifteen years, digital signal processing has been an extremely active and dynamic 
field. Advances in integrated circuit technology and in processor architecture have greatly enlarged the 
scope of the technical areas to which digital signal processing techniques can be applied. Research in 
fundamental signal processing techniques and algorithms has led to dramatic improvements in the 
efficiency of signal processing systems. 


An important facet of the progress in digital signal processing has been the development of algo- 
rithms and their embodiment in computer programs, both for the execution of processing operations on 
signals and for the design of signal processing filters and systems. The purpose of this book is to make 
widely available, in a directly usable form, a comprehensive set of computer programs useful for digital 
signal processing. In addition, the book serves as an outlet for excellent programming effort and 
enables authors of programs to receive credit for their work. 


The programs have been carefully selected to cover a broad spectrum of digital signal processing 
applications and design techniques. The programs are categorized into eight chapters, and separate 
summaries (authored by chapter editors who were also asked to make final checks on the documenta- 
tion and code) are provided with each chapter. The first chapter focuses on the Discrete Fourier 
Transform (DFT) and presents a variety of Fast Fourier Transform (FFT) and related algorithms. 
Chapter 2 includes algorithms for the periodogram and correlation methods of power spectrum estima- 
tion and for coherence and cross spectrum estimation. A program for high speed, FFT-based convolu- 
tion is presented in Chapter 3. Chapter 4 presents several algorithms related to the linear prediction 
techniques of signal processing, including the autocorrelation, covariance, and lattice methods. The 
design and synthesis of Finite Impulse Response (FIR) digital filters are the subjects of Chapter 5. 
Algorithms for optimal (minimax), windowed, and maximally-flat filter designs, and a design program 
which incorporates finite-word-length effects, are included. Chapter 6 presents a comprehensive set of 
programs dealing with the design and synthesis of Infinite Impulse Response (IIR) digital filters. The 
first program in that chapter includes most of the classical filter design techniques as well as considera- 
tion of finite word length issues such as pairing and ordering in a single but modular package. The 
other programs in that chapter are specialized either to the finite word length design problem or to filter 
design based on iterative optimization. Chapter 7 deals with cepstral and homomorphic algorithms, 
with specific attention to the difficult problem of phase calculation in a homomorphic system. Finally, 
Chapter 8 presents several programs for interpolation and decimation, the fundamental operations 
necessary for changing sampling rates in a digital system. Included are multistage implementations and 
sampling rate conversion by rational ratios. 


The book is the culmination of a project undertaken in early 1976 by the Digital Signal Processing 
Committee of the IEEE Acoustics, Speech, and Signal Processing Society. In contrast to previous 
books in the IEEE Press reprint series, this book consists largely of material which has not been pub- 
lished before. In addition, a unique and complex set of issues regarding the publication of computer 
programs had to be dealt with. These issues included: (1) clarity and uniformity of documentation and 
program presentation; (2) independent review and verification of all programs; (3) portability and 
machine-independence of code; and (4) availability of computer-readable source programs. The project 
has therefore been a very ambitious and time-consuming one for the committee, for the program con- 
tributors, and for the reviewers. 


In order to assess the feasibility of the project, the committee distributed to the digital signal pro- 
cessing community in early 1976 a description of the proposed project and a request for descriptions of 
candidate programs. The distribution of this request was carried out both through direct mailing and 
through handouts at the 1976 International Conference on Acoustics, Speech, and Signal Processing. 
In a short time, enough very promising candidate programs had been identified to convince the com- 
mittee that the book was not only feasible, but potentially a landmark contribution to the digital signal 
processing field. In addition, a reasonable organization of the material into the eight sections now 
included in the book began to emerge. 


Because of the large effort to be required of program contributors and of reviewers, it was deemed 
desirable to pre-screen program candidates based on the offered descriptions, and invite submission of 
only those programs judged most likely to fit our rather stringent requirements. In cases where two 


programs were offered to accomplish the same functions, a choice was generally made based on which 
of the programs was closer to publishable form in terms of documentation, portability, verification 
through wide previous use, or other similar criteria. 


After this pre-screening, invitations were sent to prospective authors to submit full code and docu- 
mentation for their programs. It was required of the authors that all programs meet the following con- 
ditions: 


(1) complete user documentation must be included; 
(2) the program must be in FORTRAN and conform to ANSI FORTRAN standards; 


(3) the code should be fully commented and debugged, and as clear and understandable as pos- 
sible; 


(4) the code should be transportable and machine-independent; 


(5) comprehensive test examples with corresponding input and output should be supplied. 


Of course, in addition to printouts, a computer-readable source of all code had to be provided by the 
authors. This led to the rather onerous task of collecting the submitted tapes and cards, reading them 
into a central filing system, and distributing them to reviewers in one of several possible formats. 


The committee viewed it to be an essential part of the project that computer-readable source 
material for the programs be made available to readers of the book. The collection of all programs into 
a central file was clearly necessary to achieve this. In addition, with the cooperation of Reed Crone of 
IEEE Press and Jack Fraum of the IEEE Service Center, we were able to arrange for IEEE to offer the 
service of reproducing and distributing, upon request and at a nominal cost, a tape of all programs in 
the book. For information on how to order such a tape, contact the IEEE Press, 345 East 47"" St., New 
York, NY 10017. It is worth emphasizing that this is a new service which IEEE has not offered in the 
past, and which can perhaps be utilized again in other projects. 


As authors submitted programs, the review process was able to proceed. Each reviewer was asked 
to act from points of view (1) of a highly critical program user, and (2) of an editorial reviewer of 
journal-quality technical material. The reviewer was asked to read the program description, compile the 
program, and attempt to run the test examples described as well as other test examples considered 
appropriate. The review process was an open one, with reviewers frequently consulting directly with 
authors on points of difficulty. In most cases it was found that the material submitted by authors was 
excellent and well-debugged, and the review process went quite smoothly. However, in some cases 
serious difficulties, particularly in the area of program portability, were uncovered and corrected in the 
review process. At the end of the book, we have identified the reviewer (s) as well as the computer (s) 
and operating system(s) on which each program was tested. In addition to giving credit for the efforts 
of the reviewers, this serves to identify for readers at least one successful user of each program besides 
the author(s). 


Program portability was the objective of major importance in this project. One successful review was 
not considered sufficient to insure that this goal was met; therefore an extensive parallel effort in pro- 
gram standardization was undertaken. The standardization procedures are described in detail in the 
standards section. 


Much care has been taken to insure the accuracy, Clarity, and uniformity of the printed material in 
the book. All listings of program code have been printed directly from the centrally-collected 
computer-readable source material, with particular attention to the selection of a readable type font. 
This was considered necessary to avoid either direct photoreproduction of the assorted type fonts pro- 
duced by authors or the inevitable errors which would be caused by attempting to transcribe and typeset 
the listings. The authors’ manuscripts documenting their programs were typed into a computer system, 
after which final editing and correcting were carried out. All the printed material in the book was then 
produced by computer-driven phototypesetters. 


Although this has been a project of the full Digital Signal Processing Committee as listed on the title 
page, a number of outstanding individual contributions deserve special mention here. The idea for this 
project was first presented to the committee by Ronald Crochiere in early 1976, and the project was 


initiated under the committee chairmanship of Alan Oppenheim. Crochiere was responsible for the ini- 
tial solicitation and collection of candidate programs from the digital signal processing community. 
Clifford Weinstein took over as committee chairman in September 1976 and since that time served as 
general coordinator for the program book project. 


The task of collecting submitted tapes and cards from the authors, reading them into a central file, 
and distributing them for review in specified formats, was initially carried out by Howard Helms and 
later taken over by Marie Dolan. Helms was also instrumental in arranging for IEEE reproduction of 
the program tape for users. To insure the standardization and portability of the programs, a standardi- 
zation subcommittee including James Kaiser, Dolan, Carol McGonegal, Lawrence Rabiner, and Jose 
Tribolet was made responsible for detailed specification and execution of the standards imposed on the 
programs in this book. The typing of author manuscripts into a computer file was carried out by Penny 
Blaine and Carmela Patuto. Art editing and page layout were skillfully handled by Madeline Wilson. 
Kaiser coordinated the handling of manuscripts and arranged for the phototypesetting of the book. 
Much of this phototypesetting was done by Yourdon, Inc., New York City. The subcommittee of 
Kaiser, Dolan, McGonegal, and Rabiner took charge of tracking and carrying out the many details 
required to bring the book into final form. 


Special thanks are due to Marie Dolan and Carol McGonegal, who joined the committee for the 
express purpose of participating in the program book project and who put forth enormous and very pro- 
ductive effort in seeing it through to completion. We would like to acknowledge the excellent coopera- 
tion and assistance provided to this project by IEEE Press Managing editor Reed Crone, and the specific 
efforts of Jack Fraum in arranging for IEEE reproduction of the program tape. Finally, the Committee 
must thank the program authors, reviewers, and chapter editors for the large amounts of time and 
effort they contributed to this project. 
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Standards 


In putting together this collection of programs on digital signal processing, it was immediately recog- 
nized that users would be trying to run the different programs on a wide variety of computing machines 
and under several types of operating systems. For this reason FORTRAN was chosen as the principal 
language and a great deal of attention was given to the problem of program portability. The goal was 
for each program to compile and execute, according to its defined performance criteria, on a wide range 
of computers. Adopted was the PORT Mathematical Subroutine Library approach toward portability of 
Fox, Hall, and Schryer [1]. That is, all programs “had to be portable in the IFIP sense*, with the 
exception that the three machine-constant defining functions had to be particularized to the host com- 
puter once, at installation". The techniques used in the PORT library and in the programs in this book 
to make them easily portable were: (1) the programs are written in a subset of ANSI FORTRAN, and 
(2) the target environment is specified in terms of machine-dependent parameters. In addition to por- 
tability, program readability was given consideration in this collection of programs. The goal was to 
enhance the readability of the program code by standardizing the format of the programs. Following is 
a discussion of the techniques used to make the programs portable and enhance their readability. 


Language 


The programs were restricted to the particular portable subset of ANSI FORTRAN known as 
PFORT (Portable FORTRAN) [3]. The PFORT verifier program |3] checks that individual program 
units conform to the American National Standard FORTRAN [4]; it also checked that the inter- 
program-unit communication through the use of COMMON and argument lists is consistent with the 
standard. During the review each program was passed through the PFORT verifier to guarantee its 
adherence to this language requirement. 


Machine-Dependent Quantities 


All machine-dependent quantities (I/O device codes, machine constants, etc.) are specified by the 
ILMACH, RIMACH and DIMACH FORTRAN function subprograms used in the PORT [1,5] library. 
Table 1 is a list of all the machine-dependent quantities which are specified by these routines. They are 
delivered by means of a single integer argument indicating the particular quantity desired. Complete 
listings of IMACH, RIMACH and DIMACH are given in the Appendix. Any program in this book 
which is not truly portable, that is, which uses machine-dependent quantities not given in Table 1, is 
however still transportable in the IFIP sense. Additional machine-dependent quantities have been kept 
to a minimum; instructions for changing their values are documented in the program code. 


To move the programs to a new environment, only the DATA statements in the ILMACH, 
RIMACH and DIMACH subprograms need to be changed. Values are provided in these routines for a 
number of different computing systems. They include the Burroughs 1700 system, the Burroughs 
5700/6700/7700 systems, the CDC 6000/7000 series, the CRAY 1, the Data General Eclipse S/200, 
the Harris Slash 6 and Slash 7, the Honeywell 6000 series, the IBM 360/370 series, the Xerox SIGMA 
5/7/9, the SEL Systems 85/86, the DEC PDP 10 (KA and KI processors), the DEC PDP 11, the 
UNIVAC 1100 series, and the VAX-11. 


To aid in adding values for a target environment not provided in these subprograms, a description of 
the integer and floating-point variables follows. This description appears in the PORT Mathematical 
Subroutine Library report [1]. 


* The IFIP Working Group (on Numerical Software) (WG2.5) has proposed, in a working draft, the following definitions [2] 

Portable. A program is portable over a given range of machines and compilers if, without any alteration, it can compile and run 
to satisfy specified performance criteria on that range. 
Transportable. In transferring a program between members of a given range of machines and compilers, some changes may 
be necessary to the base version before it satisfies specified performance criteria on each of the machines and compilers. The 
program is transportable if (1) the changes lend themselves to mechanical implementation by a processor, and (2) the 
changes, ideally, are limited in number, extent, and complexity. 
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Standards 


Integer variables: Let the values for integer variables be written in the s-digit, base-a form: 
+(x,;a5' + x,_,a57 + ++ + xa + xo) 


where 0 < x; < afor i = 0,...s—1. Then specify the base a, the maximum number of digits s, and the 
largest integer a*-1. Although the quantity a*1 can easily be computed from s and the base aq, it is 
provided because a naive evaluation of the formula would cause overflow on most machines and hence 
result in an incorrect value. (Storage of integers as magnitude and sign or in a complement notation is 
not specified since PORT subprograms must be independent of the storage mode.) 


Floating-point variables: If floating-point numbers are written in the tdigit, base-b form: 


where 0 < x% < OD for i=1,...,4 0 < x, and @nin <@ < Cmax, then for a particular machine, choose 
values for the parameters ¢, e,;,, and @,,, such that all numbers expressible in this form are represent- 
able by the hardware and usable from FORTRAN. Note that the formula is symmetrical under nega- 
tion but not reciprocation. On some machines a small portion of the range of permissible numbers may 
be excluded. Also, for 2’s complement machines, care must be taken in assigning the values. 


For each machine one must specify for real (single-precision) floating-point numbers, the base, b, 
the number ¢ of base-6 digits, the minimum exponent e,;, and the maximum exponent e,,,. For 
double-precision numbers, 5 remains the same, but 4 @pin, and @max are replaced by 7, E,j,, and Enax. 


The 16 parameters discussed above are all integers and are obtained by invoking the function 
I1MACH with the appropriate argument. The floating-point single-precision and double-precision quan- 
tities provided by the functions RIMACH and DIMACH can be derived from the given integer quanti- 
ties, but are provided for efficiency and convenience. 


The single-precision floating-point quantities provided in RIMACH are the smallest positive magni- 
tude, pomin the largest magnitude, Bmx (J—p-') , the smallest relative spacing between values, b~‘, 
the largest relative spacing between values, per? , and the logarithm of the base 5, log;9b. The relative 
spacing is | (y—x)/x |, when x and y are successive floating-point numbers. Equivalent values for the 
double-precision floating-point quantities are provided by DIMACH, with e,j,, @max, and t replaced by 
E min E max and 7. 


Portable Random Number Generator 


In addition to the machine-dependent quantities, the PORT library was the source of the portable 
random number generator, UNI. UNI, written by Alan Gross, is a uniform generator which will pro- 
duce the same sequence of values, to the accuracy of the computer, on any computer with 16 or more 
bits. UNI is implemented as a FORTRAN function subprogram and returns a single real random vari- 
ate from the uniform [0,1) distribution. A complete listing of UNI is given in the Appendix. There 
are three local variables (FIRST, CSEED in UNI and TSEED in the routine RIUNIF) which must 
retain their most recently assigned values. In order to adhere to this requirement on a Data General 
Computer, the STATIC mode must be set when UNI is compiled. 


Program Readability 


In order to enhance the readability of the programs, two steps were taken. First, the authors were 
asked to precede each program unit with a specific comment structure. This structure was designed to 
allow the reader to easily identify a program unit. Second, the programs (except for 1.7, 1.8, and 5.1) 
were passed through the POLISH program [6]. POLISH was written at the University of Colorado at 
Boulder and is intended to improve and standardize the format of FORTRAN programs. It does this by 
renumbering all statement labels, by inserting CONTINUE statements to force every DO to end on a 
unique CONTINUE statement, and by systematically spacing and indenting each statement in the pro- 
gram. 
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Table 1 


machine-dependent quantity function (argument) 
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CHAPTER 1 


Fast Fourier Transform Subroutines 
J. W. Cooley and M. T. Dolan 


Introduction 


The following is but a small sample of the many excellent Discrete Fourier Transform (DFT) pro- 
grams which were offered for inclusion in this book. Each of the programs selected for the book has 
some particular virtue in terms of ease of use, understanding, and efficiency in specific applications. 
Preceding each subroutine, there is included documentation which should be sufficient to enable the 
user to apply it. Comment cards in the subroutine listings give brief descriptions of the arguments and, 
to some extent, a description of the internal operations of the subroutine. The write-ups are complete 
and self-sufficient, so it only remains, in this introduction, to make a few remarks designed to enable 
the user to select, from among the subroutines, the one appropriate for his particular application, and to 
enable him to use it effectively. As a further aid to the user, Tables 1 to 5 described in more detail 
below, give storage requirements, execution time, and accuracy of test calculations for all subroutines. 


FOUREA — A Short Demonstration Version of the FFT (1.1) 


The subroutine FOUREA, by C. Rader, and others like it [1] have been written primarily to 
describe and demonstrate the FFT algorithm, and to be used as models for writing efficient special- 
purpose or microprocessor programs. In general, such programs are not intended to be used as opera- 
tional programs. However, in some terminal-operated systems and interpreters, where the running 
time is less significant, these programs have been used effectively because they are short and easily 
understood. 


Fast Fourier Transform Algorithms (1.2) 


These efficient radix 2-4-8 subroutines, contributed by G. D. Bergland and M. T. Dolan, are written 
with a completely different objective. These are written to produce very fast programs at the expense 
of instruction memory and program complexity. The first DFT-IDFT pair, FAST-FSST, computes 
1024-point transforms of real data in about one-fourth the time of the previous program, FOUREA, 
but requires 1940 words of instruction store on the Honeywell 6080N versus only 260 for FOUREA. 
The programs FFA-FFS do exactly the same thing but include the use of radix 8 DFT’s internally and, 
as a result, run still faster, e.g., 134 ms. vs the 156 ms. for FAST-FSST for the 1024-point transform. 
The price for this is 3210 words vs 1940 words of instruction store. The third program in this package 
is FFT842, which computes the DFT of a complex input. For a 1024-point complex transform, this 
program takes 250 ms., which is almost twice as long as the time required for FAST. However, it is to 
be remembered that FAST and FFA do the calculation for real rather than complex input. Therefore, 
the amount of computing they do is a little more than half of that done by FOUREA and FFT842. 


FFT Subroutines for Sequences with Special Properties (1.3) 


The FFT subroutines for sequences with special symmetry properties by L. R. Rabiner are 
extremely useful for taking advantage of symmetries in the data to save computation. Such symmetries 
arise very often from the inherent nature of the data, such as in the case of the autocorrelation func- 
tion, which is always real and even. Cosine series with only odd harmonics appear in certain types of 
Chebyshev approximations and sine series with odd harmonics are useful in solving linear partial 
differential equations with boundary conditions calling for zero values on one end and zero derivatives 
on the other. These are written in the style of FOUREA, i.e., to be short and easily read. Sines and 
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cosines are computed as needed, rather than using tables, and calculations are done in "DO" loops 
without special cases for the simpler computations. The reason for this approach here is that a rela- 
tively insignificant amount of time is spent in the (1.3) subroutines; most of the time is spent in the 
DFT subroutine which is called internally. In the programs given here, the (1.3) subroutines call FAST 
and FSST but, it is to be emphasized that these calls may be replaced by calls to any other available 
DFT subroutines. For example, if Nis not a power of two or if data is on some mass storage unit, one 
may want to use Singleton’s (1.4) or Fraser’s (1.5) FFT subroutines. 


The first four programs, FFTSYM, IFTSYM, FFTASM, and IFTASM, call FAST and FSST to com- 
pute what are actually cosine and sine transforms. Such programs have been generally available in pro- 
gram libraries. Some recent contributions to the DFT literature referred to in the write-up, have shown 
how to take advantage of further symmetries which correspond to the existence of only odd harmonics 
in the DFT. These novel ideas are incorporated in the subroutines whose names include the letters 
"OH", meaning "odd harmonics". For example, the subroutine FFTOHM computes the DFT of real 
data having a two-fold redundancy: the top half is equal to minus the bottom half. Thus, only N/2 data 
need be stored and internally, a DFT of only N/2 real points is computed. Thus, a saving by a factor 
of 2 in storage and almost a factor of 2 in computation results. The last four subroutines, whose names 
contain "SOH" and "AOH" use additional two-fold redundancies due to symmetry and anti-symmetry 
about 0 in the data. These result in still further reductions in storage and computing time. The com- 
puting times given in Tables 2a and 2b show that as N increases, one approaches a saving of a factor of 
2 for each two-fold redundancy in the data. If Nis not a power of 2, the subroutine call statements can 
be changed so as to call the mixed radix subroutines in the next sub-section. However, if N is not a 
power of 2, it must be a multiple of 4 for the first six subroutines and a multiple of 8 for the last four 
subroutines. 


Mixed Radix Fast Fourier Transforms (1.4) 


All of the above subroutines require that N be a power of 2, 4, or 8 since these are the simplest and 
most efficient forms of the FFT algorithm. The mixed radix FFT by R. C. Singleton permits one to use 
values of N containing other factors than 2. In general, the larger the prime factors of N, the less 
efficient will be the calculation in terms of numbers of operations per output point. 


Although the programs described above can be used for multidimensional transforms by simply 
computing the transforms for each vector of points in each dimension, the large number of subroutine 
calls would make this method quite inefficient. The second important feature of the mixed radix FFT 
is in the ease and efficiency with which it can be used for multidimensional complex and real 
transforms. One calls the subroutine once for each dimension and the subroutine computes the DFT 
for the number of points in that dimension, but, in the innermost loops, it repeats the calculation for 
all of the other dimensions. Table 2a shows that for powers of 2, FFT takes about the same amount of 
time as FFT842. The cost of the added flexibility is that FFT requires 2300 words of instruction 
storage while FFT842, with its subroutines, requires 1430. 


Optimized Mass Storage FFT Program (1.5) 


The sizes of the data arrays with which the above subroutines can be used depends, of course, upon 
the amount of core storage available. Depending upon the size of the machine, and the data arrays, it 
may sometimes be practical to keep the data array on a random access device such as a disk storage 
medium. The optimized mass storage FFT by D. Fraser makes this as easy and efficient as possible. 
The user defines the size of an available buffer store region of memory and provides subroutines which 
access the core storage where the data resides. The subroutine computes optimal size blocks and 
schedules the transfer of data between core storage and the mass storage medium in an optimal fashion 
according to algorithms which the author has designed and published. The array may have any number 
of dimensions and may be real or complex. The user will have to familiarize himself thoroughly with 
the documentation and will have to spend some effort on setting up read and write subroutines, but the 
result will be an extremely efficient program for very large Fourier transforms. If Nis not a power of 2 
or if one has two-dimensional arrays, then Singleton’s 2-dimensional mass-storage mixed radix FFT 
(1.9) may be preferable. 
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Chirp z-Transform Algorithm Program (1.6) 


The chirp z-transform program by L. Rabiner has been placed in this section because it is, in a 
sense, a generalization of a DFT. While the DFT may be regarded as an algorithm for evaluating the 
z-transform at N equidistant points on a unit circle, the chirp z-transform gives values of the z- 
transform at arbitrarily many equi-spaced points on an arbitrary spiral in the z-plane. In a sense, this is 
a generalized spectral analysis of discrete signals and plays the same role for discrete signals which the 
Laplace transform does in the case of continuous signals. The version given here uses FFT842 
described above, but, as with many other programs in this book, FFT842 may be replaced by any other 
appropriate DFT subroutine. 


Complex General-N Winograd Fourier Transform Algorithm (WFTA) (1.7) 


The complex general-N Winograd algorithm by J. H. McClellan and H. Nawab uses an algorithm 
which is quite different from the FFT. As far as the use is concerned, if Nis a power of 2, a subrou- 
tine using the FFT algorithm should be used, but, if there are important reasons to have other factors 
in N there may be as much as a 40% improvement in running time over the mixed radix DFT (1.4), 
for some values of N. There is no consistent rule for describing ranges of N-values for which one or 
the other algorithm or program is better. One must consult the tables included in the write-up. The 
WFTA subroutine with its required subroutines take quite a large amount of instruction store, as com- 
pared with any of the FFT programs. In judging time-wise efficiency, one must also consider that an 
initialization routine must be run for an N-value before computing the DFT. Therefore, the WFTA 
subroutine is in general preferable only when a number of computations with the same WN are to be 
done. 


Time-Efficient Radix-4 Fast Fourier Transform (1.8) 


The time-efficient radix-4 fast Fourier transform by L. R. Morris is an extreme example of trading 
off memory for time-wise efficiency when many DFT’s with the same WN are to be performed. One is 
not likely to find this type of program in the usual program libraries, but since this situation is particu- 
larly typical in digital signal processing, it is regarded as a valuable contribution to this book. The pro- 
gram used here was produced by Morris’s Autogen technique, which has been described in the litera- 
ture. This technique essentially takes all data-independent parts of a given program, in this case, the 
radix-4 part of Singleton’s program above, puts them in a preprocessing stage and effectively unwinds 
some inner loops, making special cases of certain simple operations. This produces customized optimal 
code for special situations. After reviewing this example, the reader may be interested in further 
optimization by contacting the author to produce other optimized forms of signal processing programs 
using the Autogen technique. 


Two-Dimensional Mixed Radix Mass Storage Fourier Transform (1.9) 


The mass-storage mixed radix 2-dimensional DFT by R. C. Singleton is essentially an adaptation of 
his mixed radix program described above, to do what Fraser’s program does. The main differences are 
that the present program is slightly easier to use, it does not require the N to be a power of 2 and it 
requires less instruction store. However, it is limited in its utility to 2-dimensional arrays and may not 
be as optimal as Fraser’s. 


Storage and Timing 


Table 1 contains a list of the user-called subroutines with sublists of the subroutines needed for 
each. Memory requirements are given for each subroutine along with totals for all subroutines needed 
by each user-called subroutine. These, of course, do not include storage required for data and other 
variables given as arguments in calling the subroutines. The run-totals include space for the sample cal- 
ling programs provided by the authors, data arrays, FFT subroutines, and system subroutines on the 
Honeywell 6080N. 


Tables 2a to 2d contain the average time in milliseconds for computing the transform and the 
inverse of the computed transform, starting with random sequences of length N=2”, where 
m=1,2,...,15, with some of the user-level subroutines. Tables 3a to 3d contain the standard deviation 
about the mean running time of each program. One may interpret this as saying that the computing 
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time should differ from the given time by less than the standard deviation with a probability of 68%. 


Accuracy 


In Table 4, the average value, over several runs, of the difference between the initial random 
sequences and the result of computing the DFT and then the IDFT are given. Table 5 gives the stan- 
dard deviation of the errors used for Table 4. According to the analysis of Kaneko and Liu [1], the 
variance of the error in the DFT will be 


of, = H(m)o2 , (1) 
where 2 is the mean of the squares of the elements of a(n), n=0,1,...,N—1, and 
H(m) = 0.5(5m—6)o2 + 0.25(25n?—51m+18) p22 (2) 


where m = log Nand w and o2 are the mean and variance, respectively, of the rounding error. From 
the definition of the DFT used in this book, we have a? = No?. 


The error in the IDFT will be the sum of two errors: one due to the error in the computed DFT and 
the other resulting from the IDFT calculation. The former error will be the IDFT of the DFT error 
Sa(n) which, by Parseval’s rule, make an additive contribution o#,/N to the variance of the error. The 
error produced by the IDFT calculation itself is given by a formula like that given above, except with 
the a’s replaced by xs. Assuming independence of all errors, we get, for the variance of the resulting 
x U)’s, 

o2, = 2H(m)c? , (3) 
With o2 = 1/3, this gives 
o 8, = (2/3)H(m) . (4) 


Assuming m to be moderately large, machines with proper rounding, i.e., with u = 0, will produce an 
error with a standard deviation of approximately 


Osx = V5m/3 0, . — (5) 


In machines which truncate instead of rounding, where uw ~ 0, the term with uw in H(m) dominates so 
that one obtains, approximately, 


5 
oa =~ ih, = 2mu (6) 


It can be seen in Tables 4 and 5 that the errors agree roughly with these estimates. 
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Table 1 


Memory Requirements on Honeywell 6080N 


with FORTRAN-Y Compiler 


FFTSUBS 


‘*Run-Total’’ includes all the FFT subroutines, the main test as prepared by authors, 
and the system subroutines required to run on the Honeywell 6080N computer. 


‘*Total’’ is the memory requirement for all the FFT subroutines in the package. 


‘*Subtotals’’ indicate memory necessary for independent subpackages. 


ey 


i: 


Subpackages 


FOUREA - A Short Demonstration Version of the FFT 


C. M. Rader 


Subroutine Name 
FOUREA 


Run-Total = 10K 


Fast Fourier Transform Algorithms 
G. D. Bergland and M. T. Dolan 


Subroutine Name 
FAST 
FSST 
FR2TR 
FR4TR 
FR4SYN 
FORD 1 
FORD2 
FFA 
FFS 
R2TR 
R4TR 
R8TR 
R4SYN 
R8SYN 
ORD1 
ORD2 
FFT842 
R2TX 
R4TX 
R8TX 


Run-Total =17K 


FAST,FSST,FR2TR,FR4TR,FR4SYN,FORD1,FORD2 
FFA,FFS,R2TR,R4TR,R8TR,R4SYN,R8SYN,ORD1,ORD2 
FFT842,R2TX,R4TX,R8TX 
FAST,FR2TR,FR4TR,FORD1,FORD2 
FSST,FR2TR,FR4SYN,FORD1,FORD2 
FFA,R2TR,R4TR,R8TR,ORD1,ORD2 
FFS,R2TR,R4SYN,R8SYN,ORD1,ORD2 
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Words 
260 


Total= 260 


Total = 6580 


Subtotals 
1940 
3210 
1430 
1150 
1150 
1770 
1800 
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= 


Subpackages 


FFT Subroutines for Sequences with Special Properties 


L. R. Rabiner 


Subroutine Name 
FFTSYM 
IFTSYM 
FFTASM 
IFTASM 
FFTOHM 
IFTOHM 
FFTSOH 
IFTSOH 
FFTAOH 
IFTAOH 


FAST,FSST-Subpackage from 1.2 


Run-Total =24K 


FFTSYM,FAST-Subpackage 
IFTSYM,FSST-Subpackage 


FFTSYM,IFTSYM,FAST-FSST-Subpackage 


FFTASM,FAST-Subpackage 
IFTASM,FSST-Subpackage 


FFT ASM,IFTASM,FAST-FSST-Subpackage 


FFTOHM,FAST-Subpackage 
IFTOHM,FSST-Subpackage 


FFTOHM,IFTOHM,FAST-FSST-Subpackage 
FFTSOH,FFTOHM,FAST-Subpackage 
IFTSOH,IFTOHM,FSST-Subpackage 

FFTSOH,IFTSOH, FFTOHM,IFTOHM,FAST-FSST-Subpackage 
FFTAOH,FFTOHM,FAST-Subpackage 
IFTAOH,IFTOHM,FSST-Subpackage 


Words 


240 
300 
210 
220 
250 
200 
180 
270 
180 
240 
1940 


Total= 4230 


Subtotals 


FFT AOH,IFTAOH, FFTOHM,IFTOHM,FAST-FSST-Subpackage 


1.4 


Mixed Radix Fast Fourier Transforms 


R. C. Singleton 


Subroutine Name 
FFT 
FFTMX 
REALS 
REALT 
SORTG 
NORMAL 
RMS 
ISTKGT 
ISTKRL 


Run-Total =35K 
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Total= 3440 


1390 
1450 
2480 
1360 
1370 
2370 
1400 
1350 
2390 
1580 
1620 
2840 
1580 
1590 
2810 
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Subpackages 

FFT,FFTMX,ISTKGT,ISTKRL-Subpackage 
FFT,FFTMX,ISTKGT,ISTKRL,REALS-Subpackage 
FFT,FFTMX,ISTKGT,ISTKRL,REALT-Subpackage 
FFT,FFTMX,ISTKGT,ISTKRL,REALS,REALT-Subpackage 


ee Optimized Mass Storage FFT 
D. Fraser 


Subroutine Name 
RMFFT 
CMFFT 
MFCOMP 
MFSORT 
MFREV 
MFLOAD 
MFINDX 
FMSUM 
MFRCMP 
MFRLOD 
MFPAR 
DMPERM 
RANMEF 
NAIVE 
MFREAD 
MFWRIT 


Run-Total =19K 


1.6 Chirp z-Transform Algorithm Program 
L. R. Rabiner 


Subroutine Name 
CZT 
RECUR 
DECUR 
FFT842-Subpackage from 1.2 


Run-Total =17K 
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Subtotals 
2300 
2560 
2670 
2930 


Total= 3840 


Words 
760 
140 
130 

1430 


Total= 2460 
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Lt Complex General-N Winograd Fourier Transform Algorithm (WFTA) 
J. H. McClellan and H. Nawab 
Subroutine Name Words 
INISHL 600 
CONST 600 
WFTA 42470 
WEILAVE 1280 
WEAVE2 1480 
Run-Total =65K Total= 46430 
1.8 Time-Efficient Radix-4 Fast Fourier Transform 
L. R. Morris 
Subroutine Name Words 
RADIX4 6900 
RAD4SB 90 
Run-Total =21K Total= 6990 
1.9 Two-Dimensional Mixed Radix Mass Storage Fourier Transform 
R. C. Singleton 
Subroutine Name Words 
FFT2T 350 
FFT2I 350 
XFR 40 
TRNSP 80 
EXCH 50 
MFREAD 50 
MFWRIT 40 
Run-Total =71K Total= 960 
Subpackages Subtotals 
FFT2T,XFR,TRNSP,MFWRIT,MFREAD,EXCH-Subpackage 610 
FFT21,XFR,TRNSP,MFWRIT,MFREAD,EXCH-Subpackage 610 
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Table 2a 
Average Time in ms. 
N  FOUREA - FAST-FSST  FFA-FFS FFT 842 
2 0.767 1.020 0.998 0.820 
4 1.510 1.510 1.100 1.060 
8 3.190 1.810 1.740 2.040 
16 6.790 2.960 2.360 3.370 
32 14.700 4.820 4.110 6.000 
64 31.200 8.770 7.370 12.000 
128 66.900 17.400 14.500 25.600 
256 143.000 35.200 29.200 52.600 
Siz 304.000 75.000 60.900 110.000 
1024 649.000 156.000 134.000 250.000 
2048 1380.000 339.000 282.000 533.000 
4096 2900.000 706.000 595.000 1120.000 
8192 6090.000 1520.000 1290.000 2470.000 
16384 12900.000 3180.000 2680.000 5140.000 
32768 27200.000 6790.000 5680.000 10800.000 
Table 2b 
Average Time in ms. 
Sequences with Special Properties 
N  FFITSYM- FFTASM- FFTOHM- _ FFTSOH- 
IFTSYM IFTASM IFTOHM IFTSOH 
2 0.225 0.213 0.202 0.211 
4 1.560 1.450 1.480 0.225 
8 2.170 1.960 2.080 2.100 
16 2.770 2.500 2.760 2.910 
32 4.380 4.000 4.190 3.780 
64 7.230 6.450 6.950 5.840 
128 13.300 12.000 12.600 9.880 
256 26.100 23.200 24.700 17.900 
512 52.300 46.500 49.500 35.100 
1024 109.000 97.400 103.000 69.800 
2048 223.000 200.000 212.000 143.000 
4096 467.000 421.000 445.000 286.000 
8192 971.000 882.000 933.000 607.000 
16384  2040.000 1860.000 1980.000 1250.000 
32768 4230.000 3870.000 4080.000 2600.000 


Free 
0.888 
1.100 
2.370 
3.110 
6.230 

11.800 

28.800 

54.700 

122.000 
261.000 
629.000 
1240.000 
2740.000 
5680.000 
12800.000 


FFT AOH- 
IFTAOH 
0.208 
0.219 
1.930 
2.680 
3.480 
5.430 
9.060 
16.700 
32.400 
64.800 
133.000 
266.000 
569.000 
1170.000 
2440.000 
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Table 2c : 
Average Time in ms. 
WFTA 
Initial Subsequent 
1.540 0.838 
2.210 1.380 
16 3.980 2.450 
30 8.490 5.330 
60 16.400 9.890 
120 34.500 20.900 
240 81.700 47.700 
504 173.000 108.000 
1008 416.000 252.000 
2520 1040.000 728.000 
5040 2360.000 1670.000 
Table 2d 
Average Time in ms. 
Radix4 
N Initial Subsequent 
16 3.640 1.550 
64 13.500 7.180 
256 62.300 37.200 
1024 295.000 188.000 
Table 3a 
Standard Deviation in ms. 
N FOUREA  FAST-FSST  FFA-FFS — FFT842 FFT 
2 0.053 0.073 0.067 0.049 0.011 
4 0.026 0.090 0.043 0.016 0.016 
8 0.034 0.035 0.043 0.120 0.066 
16 0.084 0.084 0.046 0.046 0.061 
32 0.076 0.051 0.152 0.094 0.028 
64 0.079 — 0.048 0.128 0.159 0.029 
128 0.130 0.009 0.034 0.043 0.059 
256 0.129 0.032 0.020 0.131 0.100 
ee 2 0.869 0.152 0.158 0.587 1.000 
1024 1.740 0.401 0.170 1.300 0.694 
2048 6.790 0.994 0.728 3.030 2.560 
4096 48.400 8.750 7.750 16.400 16.400 
8192 48.500 12.300 7.240 14.000 13.200 
16384 105.000 11.700 27.400 41.200 19.400 
32768 551.000 27.100 10.800 98.200 152.000 
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Table 3b 
Standard Deviation in ms. 
Sequences with Special Properties 


Section 1.0 
N  FFTSYM- 
IFTSYM 

2 0.006 

4 0.070 

8 0.045 

16 0.038 

32 0.055 

64 0.061 

128 0.045 

256 0.089 

we ¥, 0.162 

1024 0.257 

2048 0.138 

4096 4.800 

8192 9.510 

16384 17.000 

32768 45.500 


FFTASM- FFTOHM- — FEFTSOH- 
IFTASM IFTOHM IFTSOH 
0.012 0.006 0.005 
0.079 0.030 0.018 
0.031 0.034 0.096 
0.043 0.126 0.139 
0.131 0.045 0.158 
0.056 0.092 0.062 
0.186 0.026 0.091 
0.042 0.035 0.035 
0.045 0.053 0.144 
0.170 0.163 0.111 
1.730 0.258 0.185 
4.460 4.230 2.760 
10.600 11.300 6.030 
16.800 3.680 4.460 
29.600 29.000 26.400 
Table 3c 
Standard Deviation in ms. 
WFTA 
N Initial Subsequent 
4 0.197 0.049 
8 0.023 0.161 
16 0.051 0.029 
30 0.092 0.051 
60 0.292 0.139 
120 0.441 0.173 
240 0.863 0.413 
504 1.450 1.130 
1008 3.630 1.670 
2520 7.120 4.380 
5040 = 11.000 6.350 
Table 3d 
Standard Deviation in ms. 
Radix4 
N Initial Subsequent 
16 0.048 0.029 
64 0.167 0.085 
256 =6©0..819 0.446 
1024 3.000 1.500 
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FFTAOH- 
IFTAOH 
0.006 
0.007 
0.041 
0.096 
0.076 
0.067 
0.060 
0.120 
0.065 
0.156 
0.413 
0.319 
6.070 
9.620 
28.400 
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FOUREA 
0. 
0.72164E-15 
0.66613E-15 
0.37331E-14 
0.13850E-13 
0.26326E-13 
0.46768E-13 
0.30520E-12 
0.44593E-12 
0.89891E-12 
0.17115E-11 
0.36367E-11 
0.92585E-11 
0.25433E-10 
0.45252E-10 


FOUREA 
0. 
0.23264E-07 
0.15805E-07 
0.26457E-07 
0.36034E-07 
0.35128E-07 
0.33108E-07 
0.59805E-07 
0.51116E-07 
0.51318E-07 
0.50071E-07 
0.51610E-07 
0.58229E-07 
0.68241E-07 
0.64366E-07 


Table 4 

Sum of Squares Error 
FAST-FSST FFA-FFS 
0. 0. 
0.88818E-15 0.18128E-15 
0.19429E-15 0.30618E-15 
0.21272E-15 0.18501E-14 
0.91221E-14 0.47967E-14 
0.78466E-14 0.19558E-13 
0.28238E-13 0.61767E-13 
0.18360E-12 0.50912E-13 
0.15395E-12 0.20711E-12 
0.34031E-12 0.35218E-12 
0.61706E-12 0.61415E-12 
0.19966E-11 0.20315E-11 
0.40613E-11 0.56809E-11 
0.14997E-10 0.95540E-11 
0.20410E-10 0.25955E-10 

Table 5 
Standard Deviation of Error 

FAST-FSST FFA-FFS 
0. 0. 
0.25810E-07 0.11660E-07 
0.85357E-08 0.10715E-07 
0.63155E-08 0.18625E-07 
0.29244E-07 0.21206E-07 
0.19178E-07 0.30278E-07 
0.25726E-07 0.38048E-07 
0.46385E-07 0.24426E-07 
0.30035E-07 0.34836E-07 
0.31575E-07 0.32121E-07 
0.30065E-07 0.29994E-07 
0.38241E-07 0.38574E-07 
0.38565E-07 0.45611E-07 
0.52403E-07 0.41826E-07 
0.43227E-07 0.48747E-07 
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FFT842 

0. 

0.27756E-15 
0.11241E-14 
0.70499E-14 
0.13177E-13 
0.32644E-13 
0.68565E-13 
0.11421E-12 
0.26937E-12 
0.62708E-12 
0.13545E-11 
0.50954E-11 
0.12832E-10 
0.17858E-10 
0.33980E-10 


FFT842 

0. 

0.14428E-07 
0.20531E-07 
0.36357E-07 
0.35147E-07 
0.39118E-07 
0.40087E-07 
0.36584E-07 
0.39728E-07 
0.42862E-07 
0.44544E-07 
0.61090E-07 
0.68552E-07 
0.57184E-07 
0.55776E-07 


PrY 

0. 

0.40246E-15 
0.84655E-15 
0.36533E-14 
0.92981E-14 
0.62922E-13 
0.74431E-13 
0.24460E-12 
0.55185E-12 
0.14303E-11 


0.36801E-11. 


0.82930E-11 
0.14287E-10 
0.33178E-10 
0.86588E-10 


FFT 

0. 

0.17374E-07 
0.17817E-07 
0.26172E-07 
0.29525E-07 
0.54309E-07 
0.41767E-07 
0.53539E-07 
0.56863E-07 
0.64734E-07 
0.73422E-07 
0.77936E-07 
0.72333E-07 
0.77942E-07 
0.89036E-07 
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FOUREA - A Short Demonstration Version of the FFT 


C. M. Rader 


MIT Lincoln Laboratory 
Lexington, MA 02173 


1. Purpose 


This program can be used to evaluate the discrete Fourier transform (DFT), 
{X(k); k=0,1,...,N—1}, of an N point sequence of complex numbers, {x(n); n=0,1,...,.N—1}, where NV 
must be a power of two, N = 2”. It can also evaluate the inverse DFT. The DFT of {x(n)} is defined 
as | 


N-1 a OO gh 
Xk) «= F sine *= + k=0,1,..,.N -1 (1) 
n=0 


The inverse DFT is defined as 


eo, i> nk 
x(n) =— DY X(k)e app lt), Lay — 1 (2) 
N =0 
Unlike other programs in this collection, FOUREA is not highly recommended for actual use. Other 
programs are provided which compute the DFT for more diverse values of N, for multidimensional 
sequences, and with shorter running time. FOUREA is intended mainly for educational and demons- 
tration purposes. 


2. Method 


The program, which expects the input to be a complex array, is an implementation of the radix-2 
Cooley-Tukey algorithm [1]. The input is first rearranged into "bit-reversed" order by the DO 80 loop, 
then log,N stages of "butterflies" are performed by statements 90 to 130. The computation is per- 
formed "in-place". The internal data structure is represented by Fig. 5 of Ref. 2. 


3. Usage 
The program consists of a subroutine, FOUREA, called by 


CALL FOUREA (DATA,N,ISIGN). 


DATA is the name of an array of N complex numbers (or 2N real numbers, in which case 
DATA(2 * I-1) + jDATA(2 * I) is treated as a complex number.) 


N is the length of the sequence being transformed and must be a power of two. 


ISIGN is an integer which should be —1 to compute the DFT defined by Eq. (1). If ISIGN is 
+1 , Eq. (2) is used to compute the inverse DFT. Note that the normalization by N is 
included. 


The array DATA serves as both the input array and the output array, e.g., before the call to FOUREA, 


DATA holds the sequence to be transformed, whereas after the call it holds the transform. The index- 
ing is displaced such that x(n) is stored in array location n + 1 and X(kK) is in array location k + 1. 
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4. Test Problem 


To test any DFT program, several approaches are available. We have chosen to compare the output of 
FOUREA with the predicted DFT of a sequence whose DFT has a simple algebraic form. The 
sequence we are using is 


x(n) = Q". n=0,1.,...,.N—1 (3) 
and its DFT is 
X(k) = (0-Q%)/G-OW) kk = 0,1,...,N—1 (4) 
where 
jot. 
W=e N 


and Q is the complex constant 0.9 + j0.3. In the test we use N = 2°. The test program computes and 
prints out {x(n)}, using Eq. (3), and {X(k)}, first using FOUREA and then using Eq. (4). Next, 
{x(n)}, computed as the inverse DFT of Eq. (4) using FOUREA, is printed. Finally the program 
prints the maximum absolute difference between {X(k)} and the computed DFT of Eq. (3), and 
between {x(n)} and the computed inverse DFT of Eq. (4). These differences should be small com- 
pared to unity -- otherwise the program has failed. The test was run on a Honeywell 6080N computer 
in the example reproduced in Table 1. 
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Table 1 
COMPLEX INPUT SEQUENCE 
Bey 0.100000E 01 ip os 0.900000E 00 0O.300000E O00 
( 3) 0.720000E 00 0.540000E 00 ( 4) 0.486000E 00 0.702000E 00 
G39 0.226800E 00 0.777600E 00 ( 6) -0.291600E-01 0.767880E 00 
( 7) -0.256608E 00 0.682344E 00 é-~-@) =§ -435650E 00 -0.53,12758..00 
( 9) -0.553224E 00 0.352719E 00 ( 10) -0.603717E 00 0.151480E 00 
( 11) -0.588789E 00 -0.447828E-01 ( 12) -0.516476E 00 -0.216941E 00 
( 13) -0.399746E 00 -0.350190E 00 ( 14) -0.254714E 00 -0.435095E 00 
( 15) -0.987144E-01 -0.467999E 00 (- 46) 0.515569E-01 -0.450814E 00 
er 0.181645E 00 -0.390265E 00 (- $83 0.280560E 00 -0.296745E 00 
( 19) 0.341528E 00 -0.182903E 00 ( 20) 0.362246E 00 -0.621539E-01 
( 21) 0.344667E 00 0.527352E-01 ( 22) 0.294380E 00 0.150862E 00 
(234 0.219684E 00 0.224090E 00 | 28) 0.130488E 00 0.267586E 00 
( 25) 0.371637E-01 0.279974E 00 ( 26) -0.505448E-01 0.263125E 00 
( 27) -0.124428E 00 0.221649E 00 ( 28) -0.178480E 00 0.162156E 00 
( 29) -0.209279E 00 0.923965E-01 ( 30) -0.216070E 00 0.203732E-01 
( 31) -0.200575E 00 -0.464851E-01 ( 32) -0.166572E 00 -0.102009E 00 
FOUREA DFT 
{ 14)... 0.693973E 00 0, 3499728 01. (.. 2). 0..279327e 01. 0-805046r 01 
( 3) 0.940296E 01 -0.913501E 01 ( 4) 0.186645E 01 -0.383383E 01 
tS 0.113182ZE 01. -0.223416E 01 ae 0.904794E 00 -0.153463E 01 
( 7) 0.799557E 00 -0.113961E 01 ( 8) 0.739606E 00 -0.882314E 00 
« 9) 0.700862E 00 -0.698565E 00 C363 0.673576E 00 -0.558478E 00 
( 11) 0.653109E 00 -0.446245E 00 (12) 0.636991E 00 -0.352689E 00 
( 13) 0.623788E 00 -0.272086E 00 ( 14) 0.612613E 00 -0.200642E 00 
‘Se 5. 2 0.602883E 00 -0.135703E 00 C- 16) 0.594200E 00 -0.753136E-01 
(17) 0.586277E 00 -0.179492E-01 (18) 0.578900E 00 0.376517E-01 
( 19) 0.571899E 00 0.926070E-01 ( 20) 0.565136E 00 0.147983E 00 
( 21) 0.558492E 00 0.204881E 00 ( 22) 0.551859E 00 0.264522E 00 
( 23) 0.545134E 00 0.328365E 00 ( 24) 0.538214E 00 0.398257E 00 
( 25) 0.531002E 00 0.476678E 00 ( 26) 0.523404E 00 0.567132E 00 
( 27) 0.515362E 00 0.674850E 00 ( 28) 0.506926E 00 0.808101E 00 
( 29) 0.498467E 00 0.980906E 00 ( 30) 0.491389E 00 0.121921E 01 
( 34) 0.4907322 00 0.7577082 61..-( 32)- G2. 5179545 00 .0.278883E 01 
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THEORETICAL DFT 


os 


1) 


Oy OR OE OR FR FR ER FR SR SRS SR SR SE SO 
_— 
~< 
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~- 
— 
— 


29) 
377 


OC OO OO OO ON OO LD OLIV LVI LL LMP 
—_ 
~~ 


MAX DIFF BETWEEN THEOR AND FOUREA DFT 


MAX DIFF BETWEEN ORIGINAL DATA AND INVERSE DFT IS 


©::©O::.0-0) © 0 OO: © © ©.1.O' OO. O7.O 


.693973E 
.940296E 
.113182E 
.799557E 
.700862E 
.653109E 
.623788E 
.602883E 
.586277E 
.571898E 
.558492E 
.545134E 
.531002E 
.515362E 
.498467E 
.490732E 


INVERSE DFT 


-100000E 01 
.720000E 00 
.226800E 00 
.256608E 00 
.553224E 00 
.588789E 00 
.399746E 00 
.987144E-01 
-181645E 00 
.341528E 00 
-344667E 00 
.219684E 00 
Jt 1Gorer eo) 
-0. 
~D-. 
=U% 


124428E 00 
209279E 00 
200575E 00 
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(Continued) 
9490728 01 ( 2) 
.913501E 01 ( 4) 
22354168 01 {( ~6) 
.113961E 01 ( 8) 
.698565E 00 ( 10) 
.446245E 00 ( 12) 
.272086E 00 ( 14) 
.135703E 00 ( 16) 
.179492E-01 ( 18) 
.926070E-01 { 20) 
,204881E 00 ( 22) 
.328365E 00 ( 24) 
,476678E 00 ( 26) 
.674850E 00 ( 28) 
.980906E 00 ( 30) 
-157708E 01. -( 32) 
-39299908- 06: <(-2) 
.540000E 00 ( 4) 
.777600E 00 ( 6) 
.682344E 00 ( 8) 
.352719E 00 ( 10) 
-@478268-01 ( 12) 
.350190E 00 ( 14) 
.467999E 00 ( 16) 
.390265E 00 ( 18) 
.182903E 00 ( 20) 
Be assn-01 £233 
.224090E 00 ( 24) 
.279974E 00 ( 26) 
.221649E 00 ( 28) 
.923965E-01 ( 30) 
.464851E-01 ( 32) 


IS 


oO © CQ: OC Oo Oa 2 O'@ Go © 


2I9227E 01 
.186645E 01 
.904794E 00 
.739606E 00 
-673576E 00 
.636991E 00 
.612613E 00 
.594200E 00 
-578900E 00 
.565136E 00 
750 1B59E 00 
.538214E 00 
.523404E 00 
.506926E 00 
.491389E 00 
.517354E 00 


.900000E 00 
-486000E 00 
.291600E-01 
-435650E 00 
.603717E 00 
.516476E 00 
.254714E 00 
-515569E-01 
.280560E 00 
.362246E 00 
.294380E 00 
.130488E 00 
.505448E-01 
.178480E 00 
-216070E 00 
.166572E 00 


0.238E-06 


-805046E 01 
-383383E 01 
-153463E 01 
.882314E 00 
-558478E 00 
~-352689E 00 
.200642E 00 
~ 15343701 
-3765175-01 
-147983E 00 
»-264522E 00 
~-ase257Ez 00 
-296/132E 00 
.808101E 00 
~121921E. 01 
-218883E 01 


-300000E 00 
-702000E 00 
-767880E 00 
o3S/127E O00 
-151480E 00 
-216941E 00 
»-435095E 00 
-450814E 00 
-296745E 00 
-62753928-01 
-150862E 00 
-267586E 00 
-263125E 00 
-162156E 00 
eo) 328-01 
102009E 00 


0.263E-07 
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NOISUHA LYOHS AUBZA V SI SIHL ‘L96l SANNC ‘YANNAYA AG WVUDOUd AALAV 
*Z*¥*N IVOISSVIO 

FHL OL NVHL USHLVY ‘(N)ZDOI*XN OL IVNOILYOdOUd SI AWIL DNINNNY 
"(((L-£)*(L-L) ) **M*(I)VLVG)WOAS*(N/L)=(£)ULANI SI 

NOISSAYdXa ONINIASG AHL HOIHM YOA ‘WHOASNVUL ASUPANI SALNdMWOD 
OSTV WVUYDOUd ‘°*(N/(L-)LYOS*Id*Z*ISI) dXd9 = M GCNV N OL | WOU 
NOW C GNV I SYaHM ‘(((L-£)*(L-1I) ) **M* (I) VLVdC)WOAS=(C£)WHOAISNVUL 
; *LNdNI AHL ONIOVIdau ‘AVEUV 

LOdNI SHL NI GANUNLAY AYV SANIVA WHOASNVUL ‘WHOASNVUL GUYVMYOd 
WV YOd L- GNV WHOASNVUL ASUAANI NV YOd 1+ SI ISI ‘“OML JO WAMOd 
WV SI ‘N ‘HLONAT AZSOHM AVUUV XAIMWOO IVNOISNAWIG-ANO V SI Vivd 
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NVULYOA ISNV NI WHOASNVUL WAINNOT LSVA AAMNL-AAIOOO AHL 


VuuoOUUVOUOOUO? 


(ISI ‘N ‘VLVG)VaunNOd ANILNOWANS 


WHOASNVUL YAINNOd LSVA AAANL-AATIOOO SWHOANAd 
VauNnod :-aANILNOXsNs 


oo o-0 O-O 


(€°ZLa * 
‘ SI LAG ASUYSANI GNV VLVG IVNISIYO NAAMLAG JAIG XVW HLS/) LVWHOA €666 
(€°ZLa * SI LAG VAUNOT AGNV YOSRHL NAAMLAG AAIG XVW HZV/) LVWHOA F666 
55 (€666°QLNOI) ALIUM 
qq (%666‘°QGLNOI) ALIYM 


SONGYATAATIG WOWIXVW LNIUd oO 


GNNILNOO OP 
LSD = 99 
Ov OL OD (DD°AT°LOS) AI 
((I)0-(1I)d0)Sdvo = 155 
NN‘ L=I 0% Od 
°Q = 5D 


Lid *“YOSHL JO ASHAANI GNV LNdNI NAAMLAd “ddId *“XVW ALNdWOO 


SOROS 


(NN‘L=I‘(1I)OSI) (8666°QGLNOL) ALIYM 

( LIG ASYMANI WHUNOd HOZ//////1LHL) LYWHOA S666 
(S666‘GLNOI) ALIYM 

5 

Lid ASUYBANI LNIYd O 

2 
(L ‘NN £9D)VauNOA TIVO 


Oo dO LAG ASHAANI aALNdWOO 


UUD0 


ANNILNOD O€ 
aq = aa 
O€ OL OD (90°L9°Aa) az 
((I)ad-(I)5)Sadvo = dad 
NN‘ L=I O€ OG 
"0 = ad 


Oo GNV d@ NEEMLAG FONAAAAAIG XVW ANIA 


O0U0 


(NNSL=I‘S(I)OSI) (8666SQGLNOI) ALIYM 
( Lid IWOILAYOSHL HLL/) LVWHOA 9666 

(9666°GLNOL) AZLIYM 

(NN‘L=I‘(I)a@‘I) (8666°GLNOI) ALIUM 
( LAId VaUNOA HZL/) LVWHOI L666 

(L666‘QGLNOI) ALIUM 


FFT Programs 


LIG IVOLLSYOSHL AGNV LAG WauNOd LNIYd 
(L- ‘NN ‘@)VauNOdA TIVO 
VauNOT ONISN @ JO LAG ALNdWOD MON 
(M**MV-L)/(NN*¥*V-L) SI @ JO Lad 
ANNILNOD 02 
a/qd = (w)d 
(L-M)**M¥W - (°OS°L) = @ 
NN‘ L=_M 02 OG 
NN*¥*V - (°OS°L) =a 


WHOd GHSOTO NI @ dO LAG ALNdWOO 


OLA ‘L-M**WV SNIVLNOOD (M)@ °0**VW SNIVLNOO (L)d 


((9°PLaZ “(HL SEI S‘)HL ‘XZ)Z) LVWHOd 866 


(NNSL=I‘(I)AD‘I) (8666°QGLNOI) ZLIUM 


OUND 


mHoOO0 0 


2) 
2) 
2) 
2) 
e) 
6 


( BZONANOASe LNGNI XAIdWOD HbC//////1LHL) LVWHOX 6666 


(6666°dGLNOI) ALIUM 


SONSNOAS LNANI XFTaWOO LNTUa 
AQNILNOD OL 

(H)@ = (4) do 

(L-a)**v = (4) 


NN‘Z=_M OL Od 
(L)@ = (L)dd 
(Or a) eb re 
te" "es. ae 


NOILONNd LSdL SV A**V ALVYANAD 


((NOIdL)NIS-‘ (NOIdL)SOD)XIdWD = M 
(NN) LVOITdM/IdL = NOIdL 

(*L)NVLV*°S = Id 

OW**Z = NN 

(Z)HOWWLI = ALNOI 


LNVLSNOO SNIHOVW dN Las 
G = OW 


*YOUNR WVNSOUd VW ALVOIGNI ATdvdoOud AINOM 
UALNdWOD AHL JO AOWUNOOV AHL OL AHUYVdWOD ADUVI AONAUAAAIA XVW V 
90-S8€72Z°O SVM BAONAYAAAIG XVW AHL ‘NO809 TTISMAANOH V NO NOY NAHM 

VO-H68E€°O SVM AONSUSATIG XVW AHL ‘OLE WHI NV NO NOY NAHM 

*SAVM OML GHLNdWOD LAG NEAMLEIA AONAUAAAIAG LSADUYVT SI WVuDOUd JO LAdLNO 

L-NS* °° SL SQOH=I SI*¥*vV SNOILONONA NMONM JO LAG ALAdWOD OL SI GOHLAW 
ZTE=N SNHL ‘S=NW SASN ‘NW**Z=N HLONAT AZONANOAS 


woe teejyeo *(2e)e *S “a *4{SE)0 RM Sw TaHOS 


cee ee es ee ee me ee ee ee ee ee eee eae ae eee ee er eae ae ee 


Vaunod ANILNOWNS 
LSaL OL WYeSONd AHL AG GHLVAYANAD SI NOILONNA > LAGNI 


ELLZO WN ‘NOLDNIXA1 ‘AYOLVYOSVT NIOONIT LIW 
adaadva “W *o : YOHLNV 
LaNSunod :NvaSOudd NIVW 


mee me ee ee ee ee ee ee eee ee ee ee eee ee ee ei ee ae i ee ew ew ew ee eee eae ee 


xipuoddy 


o 
2) 
e) 


OO © O00 


Two OOO 00 0 


Ov0 OG@OU UU OU? 


1.1-4 


FOUREA 


Section 1.1 


ana 
NYNLad OI 
GANNILNOO OSL 
N4/(I)VLvd = (1I)Vivd 
N‘L=I OSL OG OFPL 


N/L AG LAdLNO ATIMILIANW -- L=ISI -- SNVUL ANI wWOd O 


OvL ‘OVL ‘O9L (ISI) 4I O€L 
06 OL OD 
daLSI = XvWW 

AQNILNOD 021 


SNNILNOOD OLL 
dWwaL + (I)ViLVvd (I) VLva by 
dWaL - (1)VLvVd = (f£)Viva 
(C)WLVG*M = dWaL 
XVWN + I= C 
daLSI‘N‘W=I OLL OG 
((VWLAHHL)NIS‘ (VLHHL)SOD)XIdWD = M 
(XVWW) LVOITA/((L-W)*ISI)LVOId*Id = VLGHL 
XVWWSL=W OZL Od 
XVWW*Z = d@LSI OOL 
O€l SOEL SOOL (N-XVWW) JI 06 
L = XVWW 


2) 
SHITAMGALLNG AHL ALNdWOO MON O 
2) 


ANNILNOO 08 


Wt+tere=rc OL 
0S OL OD 
Z/(L+W) = W 

W-cre=zc 09 

09 ‘OL “OL (NHC) a2 0S 

Z7/N = W hi 

MUALNNOD GHSUBARSU-LIG ‘L+Cl=Cf LNAWAIdMWI 
dWaL = (I)Vivad 

(I)vLvd = (f£)Vivda 
(C)VLVd = dWaL O€ 


‘C’LT'I dI (£)VLVd HLIM (I) VLVG SSNVHOXG 
OF o° OF OE. Cl4 2) at 


(L+ FO LNEWHOVIdSIG 
GTHL YOd LdeOXA) YIWd AASUAATU LId V 3uV C ANV I. ‘LNIOd SIHL LV 


N‘L=I 08 Od 
be eee as 


UAdNO GASHYAAGUY-LIG NI VLVG SLNd NOTILOGS SIHL 


N = Nd 
(°L)NVLV*°? Id 


GNNILNOO 02 
dOws 


UU0N0 UUD0 OUD 


UU00 


(VauNOA HOF @ AO UAMOd WV LON N HOE) LVWHOA 6666 


(6666‘°GLNOI) FLIYM 
GNNILNOO OL 
OZ OL OD (N°OU'°NN) AI 
c*NN = NN 
7 = 3 
GL‘ L=I OL OG 
L = NN 


Gt OL dN OML dO WAMOd WOU AOAHO 


(Z)HOVWLI = ALNOI 
M ‘dWaL XS IdWOO 
(L)WLVd XSIdWOO 


‘Lda NO GNSSI IvIOgddS £(L96l ANNE) OIGAV SNVUYL ASI -- Gas 
*SAVUUV IVNOISNAWIG-ANO OL YO 2@ dO SHYAMOd OL CaLOTaLSad 

LON 3uvV GNV YUALSVA NNY HOIHM NOILOATIOO SIHL NI AIEVIIVAV adv 
SWVUDOUd *“NOILVULSNOWEGC YOAX AINIVW GHANALNI SI GNV Ladd AHL dO 


OOo 


OR CRORORS 


1.1-5 


We 
ef 


Paks 4 


Lid 


Fast Fourier Transform Algorithms 


G. D. Berglandand M. T. Dolan 
Bell Laboratories 
Murray Hill, NJ 07974 


1. Purpose 

Fast Fourier analysis and fast Fourier synthesis algorithms are presented for computing both the 
discrete Fourier transform (DFT) of a real or complex sequence and the inverse DFT (IDFT) of a 
complex sequence. 
2. Method 
2.1 DFT (Real Input Sequence) 

The subroutines FAST (radix 4-2) and FFA (radix 8-4-2) evaluate the equation 


N-1 ge: 
X(k)= ¥x(nye *® (1) 
n=0 
where x(n) is real, X(k) is complex, and k=0,1,...,N/2. Only (N/2+1) complex DFT values need be 
computed and stored because x(n) is real and by symmetry 
X(N—k)=X (k) (2) 
for k=0,1,...,N/2 and * denoting the complex conjugate. 


The algorithms preserve the order and symmetry of the original Cooley-Tukey algorithm [1] but 
eliminate the operations which are redundant for real series, thus effecting a two-to-one reduction in 
the calculations and storage requirements [2]. 


The radix 8-4-2 routine performs as many base 8 iterations as possible and then performs one base 4 
or a base 2 iteration, if necessary. Similarly, the radix 4-2 routine performs as many base 4 iterations as 
possible and then performs one base 2 iteration, if necessary. Thus, they can be used for any value of 
N which is an integral power of 2. 


2.2 DFT (Complex Input Sequence) 


The subroutine FFIT842 (radix 8-4-2) evaluates (1) for a complex input sequence 
x(n), 0<n<N-—1, and for O0<k<N-1. With N factored into powers of 8, 4, and 2, the summed pro- 
ducts are computed iteratively very efficiently by using equations which extend the recursive equations 
of the Cooley-Tukey algorithm [3,4]. 


2.3 Inverse DFT (Real Output Sequence) 
The subroutines FSST (radix 4-2) and FFS (radix 8-4-2) evaluate the equation 


f: N=! joe nk 
x(n)=7- bi) X (ke - (3) 
k=0 


for n=0,1,...,N—1. 


These subroutines use the fact that X(k) has complex conjugate symmetry around k=N/2 in order 
to evaluate (3) in an efficient manner from the (N/2+1) complex values of X(k). 
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Effectively, these inverse algorithms are a specialization of the Sande-Tukey algorithm [5]. They are 
further characterized, as are all the algorithms, by performing in-place reordering and having no stored 
tables of sines or cosines. 


2.4 Inverse DFT (Complex Output Sequence) 
FFT842 computes the the inverse DFT by using the relationship 


(4) 


for n=0,1,...,.N—1 and * denoting the complex conjugate operation. Thus to compute an inverse DFT, 
the sequence X(k) is complex conjugated, a direct DFT is taken, and the resulting output is complex 
conjugated and normalized by the factor 1/N. 


3. Usage 
3.1 For a DFT (Real Input Sequence X) 


CALL FAST(X,N) (radix 4-2) 
or 
CALL FFA(X,N) (radix 8-4-2) 
where 
X Real array dimensioned to size N+2. On input, X contains the N-point sequence to be 
transformed (stored in locations 1 to N) and on output, X contains the (N/2+1) complex values 
of the transform. On output, the real components are stored in the odd locations of the array and 
the imaginary components are stored in the even locations. 


N _ Size of the input sequence. N must be a power of 2 and 2 < N <2!°. 
3.2 For a DFT (Complex Input Sequence X) 


CALL FFT842(IN,N,X,Y) (radix 8-4-2) 
where 


IN Input parameter set to 0 for a DFT. 
N _ Size of the input sequence. N must be a power of 2and2 < N < 7}. 


X Real array dimensioned to size N. On input, X contains the real components of the sequence to 
be transformed. On output, X contains the real components of the transform. On input and out- 
put, the N values of X are stored in locations 1 to N. 


Y Real array dimensioned to size N. On input, Y contains the imaginary components of the 
sequence to be transformed. On output, Y contains the imaginary components of the transform. 
On input and output, the N values of Y are stored in locations 1 to N. 


3.3 For an Inverse DFT (Real Output Sequence X) 


CALL FSST(X,N) (radix 4-2) 
or 
CALL FFS(X,N) (radix 8-4-2) 
where 
X Real array dimensioned to size N+2. On input, X contains the (N/2+1) complex values of the 
transform with the real components stored in the odd locations of the array and the imaginary 
components stored in the even locations. On output, X contains the sequence x(n), O0<n<N-—1, 
stored in locations 1 to N. 


N _ Size of the sequence. N must be a power of 2 and 2 < N < 2!°. 
3.4 For an Inverse DFT (Complex Output Sequence X) 


CALL FFT842(IN,N,X,Y) (radix 8-4-2) 
where 
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IN Input parameter set to 1 for an inverse DFT. 
N _ Size of the sequence. N must be a power of 2 and 2 < N < 2. 


4 Real array dimensioned to size N. On input, X contains the real components of the direct 
transform. On output, X contains the real components of the inverse. On input and output, the 
N values of X are stored in locations 1 to N. 


Y — Real array dimensioned to size N. On input, Y contains the imaginary components of the direct 
transform. On output, Y contains the imaginary components of the inverse. On input and out- 
put, the N values of Y are stored in locations 1 to N. 


4. Tests 


The complete FAST-FSST package includes: _ 
FAST and FSST (executive routines for DFT and IDFT) 
FR2TR and FR4TR (radix 2 and radix 4 iteration routines) 
FR4SYN (radix 4 synthesis routine) 
FORD1 and FORD? (in-place reordering routines) 


The complete FFA-FFS package includes: 
FFA and FFS (executive routines for DFT and IDFT) 
R2TR, R4TR, and R8TR (radix 2, 4, and 8 iteration routines) 
R4SYN and R8SYN (radix 4 and 8 synthesis routines) 
ORD1 and ORD? (in-place reordering routines) 


The complete FFT842 package includes: 
FFT842 (executive routine for both DFT and IDFT) - 
R2TX, R4TX, and R8TX (radix 2, 4, and 8 iteration routines) 


The test program can be used to check out the three packages of code. The same 32-point real 
sequence of random numbers is used as data for each package. The random numbers are from a uni- 
form distribution as generated by UNI (see Standards Section). The DFT and the inverse DFT are 
printed. In addition, a complex sequence of random numbers is used as input to FFT842. Complete 
output from the test program is in Table 1. 


The three packages of code and the test program appear in the Appendix. 
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.13654090E-08 
.94073198E-09 
.24131469E-08 
. 14063933E-08 
-66691706E-09 
.18720546E-08 
-14818243E-08 


TEST FFT842 WITH COMPLEX INPUT SEQUENCE 


REAL COMPONENTS OF INPUT SEQUENCE 


[ea EE ee JO) JP ae WN cam TN cas J ae FY | 


. 2a920607E 
.87455959E 
. 39889160E 
- 16605222E 
.61769668E 
.62402816E 
.46921754E 
. 11-37 S656E 


> IN eee 2 Ya an I GK com Neat i J ca 


. 76687502E 
-64464101E 
-45709422E 
-22602680E 
.26246527E 
-42238195E 
.54879178E-01 
-60751725E 00 


OO: Oo OC OO @ 


-68317685E 00 
.84746840E 00 
.23630936E 00 
-66245903E 00 
.51266762E 00 
.93970599E 00 
.51983086E 00 
.70150672E 00 


1.2-5 


SC. CS 1. C'S 


SOO & © 2 ove 


oe em eke eo ak =) 


C3) Ce) Cone oe 


-50919111E 00 
. 35396343E 00 
.13318188E 00 
~29021475E200 
.93920734E 00 
.28206823E 00 
.39682690E 00 
-88705479E 00 


.40612735E-01 
-46209662E 00 
.93344462E 00 
.12663724E 01 
.12900518E 01 
-14930917E 01 
. 70122849E-01 
-13026034E 01 


. 72163936E 00 
«1 17099695 -0 1 
.55190805E-01 
.10652234E 00 
.18219048E 01 
.87044481E 00 
.15931014E-01 
.19871484E 00 


-50919111E 00 
.35396344E 00 
.13318189E 00 
<2 502117768 -00 
.93920734E 00 
.28206823E 00 
.39682690E 00 
.88705480E 00 


~-12955597E-08 
.58739834E-09 
. 37602551E-08 
-49095674E-09 
.68834951E-08 
.58739834E-09 
.37602551E-08 
.25295450E-10 


~S0STSTITTE 00 
.35396343E 00 
.13318188E 00 
~25021175E 00 
.93920734E 00 
.28206823E 00 
.39682690E 00 
-88705479E 00 


FAST 


FFT Programs 


Table 1 
(Continued) 


IMAG COMPONENTS OF INPUT SEQUENCE 


oa oOo oOo ©: co 6 


o.0' oO @10O OO 2 © 


0.33143324E 00 0.22096146E 00 
0.36706980E 00 0.49780032E 00 
0.13391190E 00 0.11614336E 00 
0.98293080E 00 0.78848629E 00 
0.82732023E 00 0.10252799E 00 
0.72906453E 00 0.72203852E 00 
0.38462142E 00 0.96184775E 00 
0.51983388E 00 0.76180377E 00 
REAL COMPONENTS OF TRANSFORM 
0.15789569E 02 0.54187694E 00 
-0.91315009E 00 -0.15766885E 01 
-0.20202583E 01 -0.60150058E 00 
-0.39671516E 00 0.22274646E 00 
0.14023971E 01 -0.23689694E 01 
0.10693748E 01 -0.17185353E 01 
-0.12002746E 01 0.71695102E 00 
-0.10448134E-01 -0.32908173E 00 
IMAG COMPONENTS OF TRANSFORM 
0.16498260E 02 -0.12633017E 01 
-0 .95682339E 00 0.96735211E 00 
0.11171471E 01 0.51021537E 00 
-0 .66840816E 00 0.14368729E 01 
-0.10081615E 01 -0.12562806E 01 
0.15119098E 01 -0.84628310E-01 
0.49749768E 00 -0.28848102E 01 
-0.35731273E 01 -0.82990787E 00 
REAL COMPONENTS OF INVERSE TRANSFORM 
0.22925607E 00 0.76687504E 00 
0.87455961E 00 0.64464102E 00 
0.39889160E 00 0.45709423E 00 
0.16605221E 00 0.22602681E 00 
0.61769669E 00 0.26246527E 00 
0.62402817E 00 0.42238196E 00 
0.46921755E 00 0.54879181E-01 
0.11315656E 00 0.60751726E 00 
IMAG COMPONENTS OF INVERSE TRANSFORM 
0.33143324E 00 0.22096147E 00 
0.36706980E 00 0.49780032E 00 
0.13391190E 00 0.11614336E 00 
0.98293081E 00 0.78848629E 00 
0.82732023E 00 0.10252798E 00 
0.72906454E 00 0.72203852E 00 
0.38462142E 00 0.96184776E 00 
0.51983389E 00 0.76180378E 00 


SS. © © © Oo mo .© 


1.2-6 


.16861611E 
.25410408E 
.21288166E 
.36542587E 
.94326350E 
. 34003393E 
-48009393E 
. 70444433E 


.10384466E 
.42357683E 
.67052441E 
-47426426E 
. 19510589E 
-10908293E 
.12801024E 
.13489076E 


.22540791E 
.43692699E 
.13039546E 
.26590036E 
.11406250E 
1670575558 
.25071358E 
-35457445E 


.68317686E 
.84746840E 
.23630936E 
.66245903E 
-51266762E 
-93970599E 
.51983086E 
.70150672E 


.16861611E 
.25410408E 
.21288165E 
.36542587E 
.94326352E 
.34003393E 
.48009393E 
.70444433E 


oO oe © OO: O:.o 


OO. OO CO. oo 


©: 2.6 ©. © © 


-99224898E 00 
i797 79523E° 900 
- 78336013E 00 
-47690938E 00 
.68425309E 00 
.41031617E 00 
.96515422E-01 
.34020287E 00 


-41030719E 00 
.20724215E 00 
.14835396E 00 
sIES77537E 00 
.28028500E 01 
.23846828E 01 
.17169342E 01 
.20633298E 01 


.61337085E 00 
.28613902E 01 
.19500994E 00 
-14693253E 01 
.22069367E 01 
.22511049E 01 
.99921418E 00 
-16607314E 01 


.50919111E 00 
.35396344E 00 
.13318189E 00 
.25021175E 00 
.93920736E 00 
.28206823E 00 
.39682690E 00 
.88705480E 00 


.99224898E 00 
.79779525E 00 
. 78336014E 00 
.47690939E 00 
-68425308E 00 
.41031619E 00 
.96515425E-01 
.34020288E 00 
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FFT Subroutines for Sequences With Special Properties 


L. R. Rabiner 


Acoustics Research Dept. 
Bell Laboratories 
Murray Hill, NJ 07974 
1. Purpose 


The set of subroutines described in this paper computes the DFT and the IDFT of sequences with 
special properties. The special properties include: 


1. Symmetrical, real sequences 


2. Antisymmetrical, real sequences 

3. Real sequences with only odd harmonics 

4. Real, symmetrical sequences with only odd harmonics 

5. Real, antisymmetrical sequences with only odd harmonics. 
2. Method 


2.1 DFT Routines 


The subroutines for symmetrical and antisymmetrical sequences are based on the procedures 
described by Cooley, Lewis, and Welch [1]. If we call the input sequence x(n), defined for 
0 <n < N-1, where Nis a power of 2 (for this implementation), then if x(n) is either symmetrical, 
1.€., 


x(n) = x(N—-n) n =1,2,...,N/2 -1 (1) 
or antisymmetrical, i.e., 
x(n) = —-—x(N—-n) n =1,2,...,N/2 -1 (2) 


then the N-Point DFT of x(n) (call it ¥(k), k =0,1,...,N—1), can be obtained from the N/2-point 
DFT of the sequence y(n) (call it Y(k), k =0,1,...,N/2 — 1) where y(n) for symmetric sequences is 
obtained as 


y(n) = x(Q2n) + (x(2n4l)—x(2n-1)) n= 1,2,...,N/4 - 1 (3a) 
y(N/2—n) = x(n) — (x(2n41)—x(2n-1)) nn =1,2,...,N/4 -1 (3b) 
y(O) = x(0) | (3c) 
y(N/4)= x(N/2) (3d) 
and for antisymmetric sequences, y(n) is obtained as 
y(n) = xQn) + (x(2n4))—x(2n-1)) son = 1,2,...,N/4 - 1 (4a) 
y(N/2—n) = —x(2n) + (x(2n4l)—x(2n-1))) nn =1,2,...,N/4 -1 (4b) 
y(O) = 2x(1) (4c) 
y(N/4) = —2x(N/2-1) (4d) 


The relation for obtaining X(k) from Y(k) for symmetric sequences is [1,2] 
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X(k) = RelY(k)] + ; k= 12 ...NM (Sa) 
2 sin (=~ k) 
X(N/2—k) = Rel Y¥(k)] — A a er (5b) 
2 sin (=— k) 
X(N—-k) = X*(k) k =1,2,...,N/2—1 (Sc) 
X(0) = B(O) + Re[Y(0)] (Sd) 
X(N/2) = —B(O) + Re[Y(0)] | (Se) 
where 
a1 
B(O)=2 Y x(2n4t) (6) 
n=0 


and * denotes complex conjugate. 


For antisymmetric sequences, the relation for obtaining X(k) from Y(k) is [1,2] 
Rel Y(k)] 


X(k) = Im[Y(k)] — 5 k = 1,2,...,N/4 (7a) 
2 sin(— k) 
N 

X(N/2-k) = —Im{y(k)] - RelE@! po yg (7b) 

a; 

2 sin(— k) 

N 
X(0) = 0 (7c) 
X(N/2) =0 (74) 


It should be noted that for either symmetric or antisymmetric sequences, only (N/2+1) (symmetric) or 
(N/2) (antisymmetric) points of x(n) need be specified. 


For sequences containing only odd harmonics, i.e., 
X(2k)=0 all k (8) 


where X(k) is the DFT of x(n), the N-point DFT of x(n) can be obtained from the N/2-point DFT 
of the sequence y(n) (call it Y(k), k = 0,1,...,N/2 — 1), where y(n) is obtained as 


y(n) =4 sin (22 n)x(n) n=0,1,...N/2—1 (9) 
A recursion relation is used to obtain X¥(k) from Y(k). The required initial values are 
a 
: 27 
T,= > x(n)cos(— n) (10a) 
n=0 N 
N 
aay 
2 27 
T,= Y x(n)cos(— n) (10b) 
n=0 i 
n odd 
giving 
Re[X (1)] = 2(7,+T,) (10c) 
Rel X(N/2—1)] = 2(7,;—T,) (10d) 
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The initial condition 


ImLX (1)] = —RelY(0)]/2 (11) 
along with Eq (10c) gives the recursion 
ReLX (2k4+1) = Im[Y(k)] + Rel[X¥(2k-1)] =k =1,2,...,N/4 — 2 (12a) 
ImLX(2k+1)] = —RelY(k)] + Im[X¥(Qk-1)]_ k =1,2,...,N/4— 2 (12b) 
with final values 
ImLX¥ (N/2—1)] = Rel Y(N/4)]/2 (13) 


and Eq (10d) giving Re[X(N/2—1)]. It should again be noted that only N/2 points of x(n) are 
required to implement the procedure described above. 


For real sequences that are both symmetrical, and possess only odd harmonics a combination of two 
of the above procedures can be used [2], to further reduce computation. For this case a total of N/4 
values of x(n) are required. The procedure is a cascade of the procedure for symmetric sequences with 
that for odd harmonic sequences. The procedure can be performed in either order (i.e., one can begin 
by accounting for symmetry and then odd harmonics, or vice versa). Details of the implementation are 
given in Ref. 5. . 


For real sequences that are both antisymmetrical, and possess only odd harmonics another combina- 
tion of two of the above procedures can be used [5], to further reduce computation. For this case a 
total of (N/4+1) values of x(n) are required. The procedure is a cascade of the procedure for 
antisymmetric sequences with that for odd harmonic sequences. Again the procedure can be performed 
in either order (i.e. one can begin by accounting for antisymmetry, and then odd harmonics, or vice 
versa). Details of the implementation are given in Ref. 5. 


2.2 IDFT Routines 


The IDFT for sequences with the special properties discussed in Section 1 can be obtained from the 
DFT of these sequences by inverting the procedures described in Section 2.1. Throughout this section 
we denote the original sequence as x(n) and its DFT as X(k). 


For symmetrical sequences we first precompute the sample x (1), defined as 


N/2-1 Mar 
x(Q1) = X1=X(0)-X(W/2)+2 ¥ X (k)cos (=~ k) (14) 
k=1 
The complex sequence Y(k) is obtained from X (k) via the relation 
Y(O) = (X¥(0)+-X(N/2))/2 (15a) 
Rel Y(k)] = [X (kK) +X (N/2—k)]/2 k = 1,2....,N/4 (15b) 
Im[Y(k)] = [X(k)-—X (N/2-k)] sin( = k)] an OP (15c) 


The N/2-point IDFT of Y(k) is taken giving the N/2-point real sequence y(n). The sequence x(n) is 
obtained from y(n) via the recursion relation 


x(2n) = [y(n)+y(N/2—n)]/2 n= 1,2,...,N/4—1 (16a) 
x(2n+1) = [y(n)—y(N/2—n)]/2 + x(2n-1) on = 1,2,...,N/4- 1 (16b) 

with initial conditions 
| x (0) = y (0) (17a) 
x() = X1 (17b) 

and final condition 
x (N/2) = y(N/4) (17c) 
For antisymmetrical sequences the complex sequence Y(k) is obtained from X (kK) via the relation 

ImLY (k)] = [LX (k)—X (N/2—k)]/2 ko = 1,2,...,1N/4 (18a) 
RelY (k)] == [X(K)+X(W/2-K)] bin(= ky] k = 1,2,...,N/4 (18b) 
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The N/2-point IDFT of Y(k) is taken giving the N/2-point real sequence y(n). The sequence x(n) is 


obtained from y(n) via the recursion relation 


x(2n) = ly(n)—y(N/2—n)]/2 n = 1,2,....N/4—1 (19a) 
x(2n+1) = Ly (n)+y (N/2—n))/2 + x (Qn—-1) n = 1,2,...,.N/4—1 (19b) 
with initial conditions 
x (0) =0 (20a) 
x (1) = y(0)/2 (20b) 
and final condition 
x (N/2-1) = —y(N/4)/2 (20c) 
For odd harmonic sequences we first precompute the value of x (0) as 
x) = 2 “FS Relx(2k+0)] (21) 
N {= 
The complex sequence Y(k) is obtained from X (k) via the relation 
Re[Y¥(k)] = — Im[X (2k4+1) ]4+Im[X¥ Qk-1)] k = 1,2.,...,N/4— 1 (22a) 
Im[Y (k)] = Re LX (2k4+1)]—Rel[X (2k-1)] k = 1,2,...,N/4—-1 (22b) 
with initial and final values 
Re[Y (0) = —2Im[X¥(1)] (23a) 
Im[Y(0)] = 0 (23b) 
Rel Y(N/4)] = 2ImLX (V/2-1)] (23c) 
Im[Y(N/4)] = 0 (23d) 


The N/2-point IDFT of Y(k) is taken giving the N/2-point real sequence y(n). The sequence x(n) is 
obtained from y(n) via the relation 


x(n) = y(n)/14 sin (22 n)) n=1,2,...,.N/2—1 (24) 


with x(0) given from Eq. (21). 


The IDFT for symmetric and antisymmetric, odd harmonic sequences is obtained by cascading the 
procedures given above. The specific implementations used here first account for the time symmetry 
(or antisymmetry) and then use the odd harmonics property of the sequence, as was done for the direct 
transforms. Details of the implementation are given in Ref. 5. 


3. Program Description 
3.1 Usage 


The package consists of 10 subroutines; the DFT routines FFTSYM, FFTASM, FFTOHM, 
FFTSOH, FFTAOH, and the IDFT routines IFTSYM, IFTASM, IFTOHM, IFTSOH and IFTAOH. All 
the subroutines use the real input FFT subroutines FAST and FSST [3,4]. For the direct transforms, 
the user passes the required number of values of the data sequence ((N/2+1) for FFTSYM, N/2 for 
FFTASM, N/2 for FFTOHM, N/4 for FFTSOH, (N/4+1) for FFTAOH), the size of the full data 
sequence (N), and a scratch array of size (N/2+2) for all subroutines except FFTOHM which doesn’t 
require the scratch array. The subroutines return the DFT of x(n), X(k), in the same array in which 
the data is passed. The format in which X (kK) is returned is as follows: 


FFTSYM - only the real part of X(k), k = 0,1,...,N/2 is returned. 

FFTASM - only the imaginary part of X¥(k), k = 0,1,...,N/2 is returned. 

FFTOHM - only the odd harmonics of X¥(k), k = 1,3,...,N/2—1 are returned. The real part of X(k) is 
followed by the imaginary part of X(k). 

FFTSOH - only the real parts of the odd harmonics of X(k), k = 1,3,...,N/2—1 are returned. 
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FFTAOH - only the imaginary parts of the odd harmonics of X(k) k = 1,3,...,N/2 — 1 are returned. 


The FFT subroutine FAST is a real input, radix 4 and radix 2 FFT. Thus the subroutines only work 
for sequences whose length Nis a power of 2. (If a more general FFT is used, these routines will work 
for sequences whose length Nis a multiple of either 4 (for FFTSYM, FFTASM, and FFTOHM), or 8 
(for FFTSOH and FFTAOH)). 


For the inverse transforms, the user passes the required number of values of the transform (N/2+1 
for IFTSYM and IFTASM, N/2 for IFTOHM, N/4 for IFTSOH, and N/4 for IFTAOH), in the format 
returned by the direct transform routines, the size of the full data sequence (N), and a scratch array of 
size (N/2+2) for all subroutines except IFTOHM which does the transform in place. The inverse sub- 
routines return the IDFT of X(k), i.e. the original sequence, x(n), in the same array in which the data 
is passed, in the format required for the forward transforms. 


The inverse FFT subroutine FSST is a real input, radix 4 and radix 2 inverse FFT. Thus the sub- 
routines only work for sequences whose length N is a power of 2. (Again if a more general inverse 
FFT is used, these subroutines will work for values of N which are multiples of either 4 (for IFTSYM, 
IFTASM and IFTOHM), or 8 (for IFTSOH and IFTAOH). 


3.2 Description of Parameters of FFT Subroutines 
3.2.1 FFTSYM 


This subroutine is used to give the DFT for real, symmetric, N-point sequences where N can be any 
power of 2, or a multiple of 4 consistent with the FFT subroutines. The call to FFTSYM is 


CALL FFTSYM (X,N,Y) 
where 


X Array of size (N/2+1) points. On input X contains the first (NV/2+1) values of x(n). On output 
X contains the (V/2+1) real parts of the transform of the input. 


N _ True size of input sequence. 
Y Scratch array of size (N/2+2). 
3.2.2 FFTASM 


This subroutine is used to give the DFT for real, antisymmetric, N-point sequences where N can be 
any power of 2, or a multiple of 4 consistent with the FFT subroutine. The call to FFTASM is 


CALL FFTASM (X,N,Y) 
where 


X Array of size (N/2+1) points. On input X contains the first (V/2) values of x(n). On output X 
contains the (N/2+1) imaginary parts of the transform of the input. 


N _ True size of input sequence. 
Y Scratch array of size (N/2+2). 
3.2.3 FFTOHM 


This subroutine is used to give the DFT for real, N-point sequences having only odd harmonics 
where N can be any power of 2 or a multiple of 4 consistent with the FFT subroutine. The call to 
FFTOHM is 


CALL FFTOHM (X,N) 
where 


X Array of size (N/2+2). On input X contains the first (V/2) values of x(n). On output X con- 
tains the (N/2) values (complex) of the odd harmonics of the transform of the input. The com- 
plex values are stored as the real part of each harmonic followed by the imaginary part. 


N _ True size of input sequence. 
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3.2.4 FFTSOH 


This subroutine is used to give the DFT for real, symmetric N-point sequences having only odd har- 
monics where JN is any power of 2 or a multiple of 8 consistent with the FFT subroutine. The call to 
FFTSOH is 


CALL FFTSOH (X,N,Y) 
where 


X = Array of size (N/4). On input X contains the first (V/4) values of x(n). On output X contains 
the (N/4) real parts of the odd harmonics of the transform of the input. 


N _ True size of input sequence. 
Y Scratch array of size (N/4+4+2). 
3.2.5 FFTAOH 


This subroutine is used to give the DFT for real, antisymmetric N-point sequences having only odd 
harmonics, where N is any power of 2 or a multiple of 8 consistent with the FFT subroutine. The call 
to FFTAOH is 


CALL FFTAOH (X,N,Y) 
where 


X Array of size (N/4+1). On input X contains the first (V/4+1) values of x(7). On output X con- 
tains the (N/4) imaginary parts of the odd harmonics of the transform of the input. 


N _ True size of input sequence. 
Y Scratch array of size (N/4+2). 
3.4:0 FAST 


The subroutine FAST is a radix 4 and radix 2 FFT subroutine [3,4] which uses real inputs. The call 
to FAST is 


CALL FAST (X,N) 
where 


X Array of size (N+2). On input X contains the N values of the sequence to be transformed; on 
output X contains the (N/2+1) complex values of the transform of the input. 


N _ Size of the input sequence. 
3.2.7 IFTSYM 


This subroutine is used to give the inverse DFT (IDFT) for real, symmetric N-point sequences 
where N can be any power of 2, or a multiple of 4 consistent with the IFFT subroutine. The call to 
IFTSYM is 


CALL IFTSYM (X,N,Y) 
where 


X Array of size (N/2+1) points. On input X contains the (V/2+1) real parts of the transform of 
x(n); on output X contains the first (V/2+1) values of x(n). 


N _ True size of input sequence. 
Y Scratch array of size (N/2+2). 
3.2.8 IFTASM 


This subroutine is used to give the IDFT for real, antisymmetric N-point sequences where N can be 
any power of 2, or a multiple of 4 consistent with the IFFT subroutine. The call to IFTASM is 


CALL IFTASM (X,N,Y) 
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where 


X Array of size (N/2+1) points. On input X contains the (NV/2+1) imaginary parts of the transform 
of x(n); on output X contains the first (V/2) values of x(n). 


N _ True size of input sequence. 
Y Scratch array of size (N/2+2). 
3.2.9 IFTOHM 


This subroutine is used to give the IDFT for real, N-point sequences having only odd harmonics 
where N can be any power of 2, or a multiple of 4 consistent with the IFFT subroutine. The call to 
IFTOHM is : 


CALL IFTOHM (X,N) 
where 


X Array of size (N/2+2). On input X contains the (V/2) values (complex) of the odd harmonics 
of the transform of x(n); on output X contains the first (N/2) values of x(n). On input, the 
complex values are stored as the real part of each harmonic followed by the imaginary part. 


N _ True size of input sequence. 
3.2.10 IFTSOH 


This subroutine is used to give the IDFT for real, symmetric N-point sequences having only odd 
harmonics where JN is any power of 2 or a multiple of 8 consistent with the IFFT subroutine. The call 
to IFTSOH is 


CALL IFTSOH (X,N,Y) 
where 


X Array of size (N/4). On input X contains the (N/4) real parts of the odd harmonics of the 
transform of x(); on output X contains the first (N/4) values of x(n). 


N _ True size of the input sequence. 
Y Scratch array of size (N/4+2). 
3.2.11 IFTAOH 


This subroutine is used to give the IDFT for real, antisymmetric N-point sequences having only odd 
harmonics, where JN is any power of 2 or a multiple of 8 consistent with the IFFT subroutine. The call 
to IFTAOH is 


CALL IFTAOH (X,N,Y) 
where 


X Array of size (N/4+1). On input X contains the (V/4) imaginary parts of the odd harmonics of 
the transform of x(n); on output X contains the first (V/4+1) values of x(n). 


N _ True size of input sequence. 
Y Scratch array of size (N/4+2). 
3.212 FSST 


The subroutine FSST is a radix 4 and radix 2 IFFT subroutine [3,4] for real input sequences. The 
call to FSST is 


CALL FSST (X,N) 
where 


X Array of size (N+2). On input X contains the (NV/2+1) complex values of the transform of 
x(n); on output X contains the N real values of the sequence x(n). 
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N _ Size of the input sequence. 
3.3 Dimension Requirements 


The DIMENSION statement in the calling program should be modified according to the require- 
ments of each particular problem. The required dimensions of the arrays are 


X  (N/2+2) values, or (N/4+1) values, depending on the subroutine used, where N is the size of 
the data array. 


Y  (N/2+2) values, or (N/4+2) values depending on the subroutine used. 
3.4 Summary of User Requirements 


1. Specify the required number of values of x(n), the input sequence, or X(k) the 
transform of x(n). 


2. Call the appropriate FFT or IFFT subroutine. 
3. Output DFT or IDFT is obtained in the same array in which the input was defined. 


4. Test Program 


The test program FFTSUBT tests each of the 10 subroutines in order. The program first asks the 
user to specify the value of N (the sequence length). It then creates an N-point random, real, sym- 
metrical sequence, and then computes and prints out both the sequence and the entire (N/2+1) point 
complex FFT with the real part of each FFT component preceding the imaginary part. The first 
(N/2+1) time samples of x(n) are then passed on to the subroutine FFTSYM and the (N/2+1) points 
of the output (the real parts of the FFT) are printed. The output of FFTSYM is then passed to IFT- 
SYM and the resulting (N/2+1) values of x(n) are printed. 


The program next creates a new N-point random, real, antisymmetrical sequence and computes and 
prints out both the sequence and the entire (N/2+1) point complex FFT. The subroutine FFTASM is 
used on the first (N/2) time samples and the (N/2+1) points of the output (the imaginary parts of the 
FFT) are printed. The output of FFTASM is then passed to IFTASM and the resulting (N/2) values 
of x(n) are printed. 


The program next creates a new N-point random, real sequence with only odd harmonics. This 
sequence is created directly in the frequency domain by randomly specifying values for the complex 
odd harmonics, and setting the even harmonics to zero. Again the entire (N/2+1) complex values of 
the FFT of the sequence are printed. The real sequence is obtained as the inverse FFT of the DFT 
(obtained using FSST [3,4]) and the N time values are printed, and the first (N/2) time samples are 
passed on to the subroutine FFTOHM and the (N/4) complex odd harmonics are printed. The output 
of FFTOHM is then passed to IFTOHM and the resulting (N/2) values of x(n) are printed. 


Next, the program creates an N-point random, real, symmetric sequence with only odd harmonics. 
This sequence is again created directly in the frequency domain. The entire (N/2+1) complex values 
of the FFT of the sequence are printed. The time sequence is obtained using the inverse FFT subrou- 
tine FSST and the N values of x(n) are printed. The first (V/4) time samples are passed on to the 
subroutine FFTSOH and the (N/4) real values of the odd harmonics are printed. The output of 
FFTSOH is then passed to IFTSOH and the resulting (N/4) values of x(n) are printed. 


Finally the program creates an N-point random, real, antisymmetric sequence with only odd har- 
monics. This sequence is again created directly in the frequency domain. The entire (N/2+1) complex 
values of the FFT of the sequence are printed. The time sequence is again obtained using the inverse 
FFT subroutine FSST and the N values of x(n) are printed. The first (V/4+1) time samples are 
passed on to the subroutine FFTAOH and the (N/4) imaginary values of the odd harmonics are 
printed. The output of FFTAOH is then passed to IFTAOH and the resulting (N/4+1) values of x(n) 
are printed. 


The program then loops back and asks the user to specify a new value of N to repeat the tests. A 
value of N = 0 terminates the program. 
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The code for the test program and the subroutines FFTSYM, IFTSYM, FFTASM, IFTASM, 
FFTOHM, IFTOHM, FFTSOH, IFTSOH, FFTAOH, and IFTAOH is given in the Appendix. Table 1 
gives output of the test program for a value of N = 64. 
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TESTING N= 


ORIGINAL SYMMETRIC SEQUENCE 
.27074E 00 
-14464E 00 
-26369E 00 
-24979E 00 
.12403E 00 
-44512E 00 
.20151E 00 
eos 7B OO 
-43971E 00 
23753E 00 
iega908 00 
-14604E 00 
-18318E 00 


64 POINT FFT 
.97649E 00 
.89407E-06 
-49675E 01 
cle? ae 
.50420E 00 
.47684E-06 
.25434E 01 
.96858E-07 
. 09928 07 
-65565E-06 
wee 1 SE 01 
.18887E-05 
.18869E 01 
-OOO000E 00 


.38705E 00 
-37456E 00 
.42906E-01 
.16246E 00 
-43921E 00 
s0/B2E-01 
107 928-00 
.38684E 00 
ed f 29e RO 
-12667E-01 
-27397E 00 
-10111E 00 
»v1910E-02 


OF SYMMETRIC 
.0OQ0000E 00 
-38044E 01 
.41672E-05 
-64130E 00 
.20862E-05 
-84500E 00 
. 29653E-05 
-16805E 01 
-10729E-05 
-19977E 01 
.13523E-05 
22661E 01 
AASseon- Oo 


Table 1 


64 RANDOM SEQUENCES 


.26687E 00 
.34747E 00 
.36682E 00 
.11770E 00 
. 77618E-01 
-19831E-01 
-16857E 00 
.19831E-01 
. 7761 8E-01 
-11770E 00 
.36682E 00 
.34747E 00 
.26687E 00 


SEQUENCE 

.16830E 01 
.33458E-06 
.31376E 00 
-QO000E 00 
-71593E 00 
-14163E-05 
.64029E-01 
.14342E-05 
.52683E 00 
-99465E-06 
.36140E 01 
-14305E-05 
.14484E 01 
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.18318E 00 
.14604E 00 
passes UO 
tot oan OO 
.43971E 00 
~10317E O0 
»20151E 00 
-44512E 00 
.12403E 00 
.24979E 00 
.26369E 00 
.14464E 00 
.27074E 00 


<12703E-05 
.56949E 00 
.53644E-06 
40633E 01 
-15087E-05 
.35235E 00 
.OQ000E 00 
»2ISIOE 04 
.23060E-05 
-259730E 01 
.32/83E-05 
.26874E 00 
.67428E-06 


.91910E-02 
-10111E 00 
27397E 00 
.12667E-01 
~2t/93EB 00 
.38684E 00 
/ S2E 00 
.30782E-01 
-43921E 00 
.16246E 00 
.42906E-01 
.37456E 00 


-20771E 01 
.83447E-06 
-20166E 01 
-91270E-06 
.11459E 01 
-I9232E-06 
-40404E 00 
.82026E-06 
. 38979E -00 
-QO000E 00 
.67098E 00 
.18843E-05 
.33604E 01 
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OUTPUT OF FFTSYM 
-.97649E 00 
-.49675E 01 
-.50420E 00 - 
-.25434E 01 
=~. 105525 -61 

I2Z219E 01 
-18869E 01 


OUTPUT OF IFTSYM 
~-38705E 00 - 
-37456E 00 

-.42906E-01 - 
-16247E 00 - 
-43921E 00 

-.30//6E-01 = 
-10752E 00 


-16830E 01 
-31375E 00 
» 71593E 00 
-64028E-01 
-52684E 00 
-36141E 01 
-14484E 01 


-27074E 00 
-14464E 00 
-26368E 00 
-24979E 00 
.12403E 00 
-44512E 00 
sZ01STE: 80 


Table 1 
(Continued) 


-20771E 01 
-20166E 01 
-11459E 01 
-40406E 00 
- 38979E 00 
-67099E 00 
-33604E 01 


-26687E 00 
-34747E 00 
-36682E 00 
-11770E 00 
. 7761 8E-01 
-19838E-01 
-16857E 00 


ORIGINAL ANTISYMMETRIC SEQUENCE 


-OOOOOE 00 -. 
=. 21999802 34% 
-28336E 00 
-32732E 00 
-22204E 00 
-19906E-01 
-15980E 00 
-19834E-01 
-89684E-01 
-44326E 00 
-28849E 00 
-36609E 00 
-49225E 00 


-.28712E 00 

- .23091E-01 
- 22906E. 00 
-46185E 00 - 
-20444E 

-.26180E 00 - 
-11538E 00 

-.18425E 00 - 
-13457E 
-38386E 00 
-t3293E 00: = 


64 POINT FFT OF ANTISYMMETRIC SEQUENCE 
-89407E-06 
s255075 0A 
. 53644E-06 
-90270E 00 
-50068E-05 
-44543E 00 
-O0000E 00 
-17832E 01 
-19073E-05 
-26390E 00 
-45896E-05 
-14768E 01 
-10133E-05 


-OO0O000E 00 
—0215E 04 


»2I458E-05 — 
-OOQO00E 00 
Je soe OT 
-17285E-05 
-66126E 01 
-83447E-06 
-17614E 01 
-21458E-05 
-82171E 00 
-48056E-06 
-22052E 01 


-.24839E 01 
.25034E-05 
.29557E 00 
.48280E-05 
.32414E 01 

-~.71153E-06 

-. TS702E 1 = 

-.33379E-05 - 
.18796E 01. - 

-.89407E-06 - 
.00000E 00 


OUTPUT OF FFTASM 
-0O0000E 
-.10940E 01 - 
-32936E 01 - 
-66127E 01 
-17614E 
-204.) (ee pe = 
~~ee200“28 Ul. = 


OUTPUT OF IFTASM 
-OOOO00E 00 - 
-.22032E-02 - 
~.26/12E 00 
- .23091E-01 
-22906E 00 
-46185E 00 - 


27904E 00 
24590E 00 


-Q0000E 00 
.77486E-06 


10940E 01 


-18801E 01 
-73940E 00 
-24539E 01 
-81999E 00 
-18068E 01 
-30256E 01 
-44018E 00 


-27904E 00 
-24590E 00 
-28336E 00 
-3IZ732E 00 
-22204E 00 
-19906E-01 
-20444E 00 -. 


15980E 00 


+ 331508 00 
-29780E 00 
-48293E 00 
-39747E 00 
-15997E 00 
-40348E 00 
-OO0000E 00 
-40348E 00 
-15997E 00 
-39747E 00 
-48293E 00 
-29780E 00 
-33138E 00 


mueke Fae. a 
-24839E 01 
-29558E 00 
-32414E 01 
-15702E 01 
-18796E 01 
-OO000E 00 


-33138E 00 
-29780E 00 
-48293E 00 
-39747E 00 
-15997E 00 
-40349E 00 


1.3-10 


-38044E 01 
-64131E 00 
-84498E 00 . 
-16805E 01 
~-19977E 01 
-22661E 01 


-18318E 00 
-14604E 00 
-33394E 00 
-23753E 00 
-43971E 00 
-10317E 00 


~-49225E 00 
-36609E 00 
-28849E 00 
-44326E 00 
-89684E-01 
-19834E-01 
-15980E 00 
-19906E-01 
-22204E 00 
-32732E 00 
-28336E 00 
-24590E 00 
-27904E 00 


mee tE. 04 
-35390E-06 
»73939E 00 
-23842E-06 
-24539E 01 
-99605E-06 
-81998E 00 
-14901E-05 
-18068E 01 
-OOO00E 00 
-30256E 01 
-53644E-06 
-44018E 00 


sae00 7 DF 
-90271E 00 
-44543E 00 
-17832E 01 
-e0oven DY 
-14768E 01 


-49225E 00 
-36609E 00 
-28849E 00 
-44326E 00 
-89687E-01 
-19834E-01 


-56949E 00 
-10633E 01 
-35234E 00 
-31990E 01 
-25730E 01 
-26874E 00 


-91908E-02 
-10110E 00 
-27397E 00 
-12673E-01 
-21793E 00 
-38684E 00 


-13293E 00 
-38386E 00 
-13457E 00 
-18425E 00 
-11538E 00 
-26180E 00 
-20444E 00 
-46185E 00 
-22906E 00 
-23091E-01 
-28712E 00 
-21999E-02 


-53644E-06 
-31046E 00 
-OO0O000E 00 
-36229E 01 
- 8381 9E-06 
IVITIE 00 
-28014E-05 
-10330E 01 
-15497E-05 
-14661E 01 
-23842E-06 
-45427E 01 
-OOO000E 00 


-31046E 00 
-36229E 01 
-50970E 00 
-10330E 01 
-14661E 01 
-45427E 01 


-toe95b 00 
-38386E 00 
-13457E 00 
-18425E 00 
-11538E 00 
-26180E 00 


Chapter 1 


Section 1.3 


64 POINT FFT 
-OO0O000E O00 
.QO000E 00 
. 78124E-01 
.12580E 00 
.OO0Q000E 00 
-OO000E 00 
-11457E 00 
.25726E 00 
-QOOO0E 00 
-QO0000E 00 
-24664E 00 
-12919SE GO 
-OQ0O000E 00 
-OO0000E O00 


Table 1 
(Continued) 


OF ODD HARMONIC SEQUENCE 


.QO0000E 00 
-24817E 00 
. 5861 8E-01 
-OO000E 00 
.-QQ000E 00 
s3072Z7E 00 
.32150E-01 
-OQO000E 00 
-OQ0000E 00 
-39448E 00 
.36574E 00 
-OO000E 00 
.QQ0000E 00 


392398 -01 


.15425E 00 
.00000E 00 
.00000E 00 
.48475E 00 
.20840E 00 
.00000E 00 
.00000E 00. 
.48588E 00 
.26213E 00 
.00000E 00 
.00000E 00 


~,£69188 090 


ORIGINAL ODD HARMONIC SEQUENCE 
-./1683E-02 


OUTPUT OF FFTOHM 
#3923 9E-01 
55861 9E-01 
-48475E 00 
aan to1E-O4 
-48589E 00 
.36574E 00 
.22918E 00 


OUTPUT OF IFTOHM 
.20453E-01 
-29470E-01 
.20514E-01 
-11459E-01 
-53001E-01 
.87986E-01 
-42196E-02 


.20453E-01 
.29470E-01 
.20514E-01 
-11459E-01 
. 93000E-01 
“8 /985E-04 
-42196E-02 
.13413E-01 
~278158-01 
.54190E-01 
.28193E-01 
-21689E-02 
-331-98E-01 


.39408E-01 
-15314E-01 
.10145E-01 
-119325-01 
.6841 2E-01 
-46738E-02 
.91415E-02 
2/8217E-02 
.18272E-01 
.32431E-01 
.19987E-01 
-81134E-01 
.23224E-01 


.24476E-01 
-27441E 00 
.32958E 00 
«LI971E -00 
.40457E 00 
-47931E 00 
~919.72E-01 


.39408E-01 
-15314E-01 
-10145E-01 
<Lte32n~O1 
.68412E-01 
-46738E-02 
.91415E-02 


27 IT 10E- 
- T0090E~ 
«tS036E= 
»22913E- 
s60175E- 
.20453E- 
-29470E- 
.20514E- 
-11459E- 
i 230808 - 
.87985E- 
.42196E- 


.24817E 
-12580E 
.30727E 
-25726E 
. 39448E 
tee 1OE 


. /1683E- 
9 7910E=- 
100 90F,= 
-/3036E- 
oie 13E= 
-60175E- 


02 
01 
01 
01 
02 
01 
01 
01 
01 
01 
01 
02 


00 
00 
00 
00 
00 
00 


02 
02 
01 
01 
01 
02 


- .24476E-01 
-OO000E 00 
.OQ0000E 00 
-24052E-01 
—I2958E 00 
-OOOO00E 00 
.OQ000E 00 
.30139E 00 
-40457E 00 
-OO000E 00 
.OQO000E 00 
.54238E-01 
-91972E-01 


.13413E-01 
«27815E-01 
.54190E-01 
e200 193E-01 
.21689E-02 
m2 198E=-01 
.39408E-01 
.15314E-01 
-10145E-01 
-t1T932E-0% 
.68412E-01 
.46738E-02 
.91415E-02 


-15425E 00 
«240535 -0-1 
.20840E 00 
-20:139E 00 
262135 00 
.54236E-01 


-13413E-01 
-27815E-01 
.54190E-01 
wee or — 0.1 
-21689E-02 
«33198E-01 


64 POINT FFT OF ODD HARMONIC, SYMMETRIC SEQUENCE 


-OOO0O00E O00 
.-QQ0000E 00 
224298 OY 
.QO0000E 00 
-QOQ0O00E 00 
-OO0O000E 00 
.94914E-01 
-QQ0000E 00 
-QOOO00E 00 
.QQ0O000E 00 
-46172E-01 
-OQO000E O00 
-OOO000E 00 
-QO0000E 00 


-OOO000E O00 
.14412E 00 
-OOO000E 00 
-OQQ00E 00 
-OOOO00E O00 
.34142E 00 
-OOO000E 00 
.-QQ0000E 00 
-OOOO00E 00 
~69936E-01 
-OOOO00E 00 
-OQ000E 00 
-OOQ00E 00 


-26539E 
. 00000E 
-OOQO000E 
. O0O000E 
es doy 2 
. OO000E 
-OO0000E 
. OO000E 
-44473E 
. 00000E 
-0OO0000E 
. OOO00E 
-12386E 


1.3-11 


-OOOO00E O00 
.-QO0000E 00 
-OQOOO0E QO 
-30587E 00 
-OOOO00E 00 
.OO0000E 00 
-OOO00E 00 
.14686E 00 
-QOQO00E 00 
.OQ0O000E 00 
-OOO000E 00 
~-41555E 00 
-OOQ000E 00 


-QQ000E 00 
-OOO000E 00 
.27441E 00 
-40714E 00 
-QQO000E 00 
-OOO000E 00 
«tT9T1E 00 
-47430E 00 
-QO000E 00 
-QOOO0E 00 
-47930E 00 
-41343E 00 
-QQ0000E 00 


1821 7E-02 
-18272E-01 
.32431E-01 
.19987E-01 
.81134E-01 
-23224E-01 
- 71683E-02 
/9IDE-02 
. 10090E-01 
~ (303562 -01 
<22913E-01 
-60175E-02 


.78123E-01 
.40714E 00 
.11458E 00 
.47430E 00 
.24664E 00 
.41343E 00 


. 7821 7E-02 
- 18272E-01 
.32432E-01 
wsbeao/n-07 
.81134E-01 
.23224E-01 


-OOO000E 00 
-QO0000E 00 
-26810E 00 
-QOQ00E 00 
-OQO000E 00 
-QQO000E 00 
.34192E 00 
-QQ0O000E 00 
-OOO0O00E O00 
-QO0000E 00 
fa3925-03 
-QQO000E 00 
-OOO00E 00 


FFTSYM 


FFT Programs 


ORIGINAL ODD HARMONIC, 


14629E-01 


.96356E-02 
.90984E-02 
.40847E-01 
.85548E-02 
.16822E-01 
-43949E-01 
.30324E-01 
.95453E-02 
~19091E-01 
.24814E-01 
.42268E-02 
“ta1/ae-02Z 


OUTPUT OF FFTSOH 


26539E 00 
13275E 00 
44473E 00 
12386E 00 


OUTPUT OF IFTSOH 


14630E-01 
96353E-02 
90986E-02 


- .40847E-01 


64 POINT FFT OF ODD HARMONIC, ANTISYMMETRIC SEQUENCE 
.OO0O000E 00 
.OO0Q000E 00 
-QO000E 00 
.32781E 00 
-QQO000E 00 
.QQ0O000E 00 
-OOO000E 00 
sus Jze~C1 
-OO000E 00 
.OO0000E 00 
-QO000E 00 
2391 tE 00 
-OO000E O00 
.OO0Q000E O00 


2Z2831E-C2 


.32010E-01 
eliaoae oO | 
.QO0000E 00 
Liss sen eT 
. 32010E-01 
.I2831E-02 
.75174E-02 
-42268E-02 
.24814E-01 
i Na 
.95453E-02 
.30324E-01 


.14412E 00 
.34142E 00 
.69939E-01 


.92827E-02 
.32011E-01 
ot FSSsn- 01 


-OOO000E 00 
.OO0000E 00 
-80915E-01 
.QQ0000E 00 
-OO000E 00 
.QQ000E 00 
.27775E 00 
.QQ0000E 00 
-OOO000E 00 
.OQO000E 00 
-45259E 00 
.QO0000E 00 
-OOOO0E O00 


Table 1 


(Continued) 


SYMMETRIC SEQUENCE 
-43949E-01 
, eeZ2z2E=07 
-85548E-02 
.40847E-01 
.90984E-02 
- 9e3968~+02 
.14629E-01 
- 96356E-02 
.90984E-02 
.40847E-01 
-85548E-02 
-16822E-01 
-43949E-01 


-22329E 00 
.94915E-01 
.46169E-01 


.43948E-01 


16822E-01 
85547E-02 


.OO0000E 00 
.45413E 00 
-O0000E 00 
.OO0000E 00 
.OO000E 00 
.99167E-01 
-OOO000E 00 
.OQ0O000E 00 
.OO0O000E 00 
.27765E 00 
-QO000E 00 
.QQ0000E 00 
-QO0000E 00 


.30324E-01 
-95453E-02 
2 PSUS EO 1 
.24814E-01 
-42268E-02 
Peet as a A oe? a 
J2051h-02 
.32010E-01 
Trgasu ey 
.QO0000E 00 
ee EE) ee 
.32010E-01 
foL03 1-02 


.26811E 00 
.34192E 00 
. 78589E-03 


.30324E-01 
.95451E-02 
-19092E-01 


.13362E 00 
.OO0000E 00 
.O0O000E 00 
.0O0Q000E 00 
-41614E 00 
.OQ0000E 00 
-OOO000E 00 
.OO0000E 00 
Jratok 00 
.OQQ0Q00E 00 
.OOO000E 00 
.00000E 
.28792E 00 


ORIGINAL ODD HARMONIC, ANTISYMMETRIC SEQUENCE 


OUTPUT OF FFTAOH 


-OOO0O00E 00 
2 eo2e C1 
.27449E-01 
-65640E-01 
-68284E-02 
-24136E-02 
.38045E-01 
-40039E-01 
~seseen-Ue 
.61149E-02 
-20351E-01 
.38762E-01 
.18682E-01 


13362E 00 
41614E 00 
siaton UG 
28792E 00 


30825E-03 


-20747E-02 
.cs2208- 01 
-24044E-01 
-2ena0E-07 
-20747E-02 
. 30822E-03 
-18682E-01 
. 38762E-01 
.20351E-01 
.61149E-02 
39 S92E-U2 
-40039E-01 


.45413E 00 
<49 1635-071 
-27765E 00 


38045E-01 


.22736E-02 
.68283E-02 
.65640E-01 
.27449E-01 
.31662E-01 
.00000E 00 
.31662E-01 
.27449E-01 
.65640E-01 
.68284E-02 
.22736E-02 
.38045E-01 


.80916E-01 
-27776E 00 
-45259E 00 


1.3-12 


-40039E-01 
<37 392E-02 
-61149E-02 
2039 1E=-01 
. 38762E-01 
-18682E-01 
. 3 0825E-03 
-20747E-02 
2i230E-01 
-24044E-01 
-23230E-01 
-20747E-02 
. 30822E-03 


.32781E 00 
8951E-01 
-239T1E 00 


.75174E-02 
~-42268E-02 
.24814E-01 
»T909TE-07 
.95453E-02 
.30324E-01 
-43949E-01 
- 16822E-01 
.85548E-02 
.40847E-01 
.90984E-02 
-96356E-02 


.30587E 00 
.14686E 00 
.41555E 00 


ctoarilE-O2 
-42265E-02 
.24814E-01 


-OO000E 00 
.00000E 00 
.OO0O000E 00 
-46997E 00 
.OO0000E 00 
.QO0000E 00 
.OO0000E 00 
.72483E-01 
.QO000E 00 
.QQ0000E 00 
-OO000E 00 
-43002E 00 
.OO000E 00 


.18682E-01 
.38762E-01 
.20351E-01 
-61149E-02 
. I 9S92ZE-02 
-40039E-01 
. 38045E-01 
.22736E-02 
-68283E-02 
-65640E-01 
.27449E-01 
~32 L662E-07 


-46998E 00 
.72483E-01 
-43002E 00 


Chapter 1 


Section 1.3 
Table 1 
(Continued) 
OUTPUT OF IFTAOH 
-OO0000EF 00 -.30849E-03 -.38045E-01 .40039E-01 .18682E-01 
.31662E-01 .20747E-02 .22732E-02 -.39391E-02 -.38763E-01 
.27449E-01 .23230E-01 68284E-02 -.617152E-02 -.20351E-01 


- .65640E-01 .24044E-01 


1.3-13 


FFTISYM 


(7666‘ZLNOL) :ALIUM 
(L4ZON*L=I‘(I)X) (€666*ZLNOLI) ALIUM 
(WSVLGd JO LNdLNO HLL) LYWwYOa 
(L866°7LNOL) ALIUM 
(A *‘N *X)WSVLda TIvoO 
ANNILNOOD 
(TVA 12S 
ZON‘ L=I 06 OG 


Chapter 1 


Laid LNIOd CON WOWd LAd NIVLEO OL WSVLda ANILNOWENS 


(7666°7LNOL) ALIUM 

(ZAN‘L=I‘(I)X) (€666‘ZLNOL) ALIUM 

(AONANOAS OIULAWWASILNV JO Ladd LNIOd HOE ‘PI * HL) LVWYOd 
N (8866°ZLNOL) ALIYM 

(N *X)LSVa TIVO 


L866 


06 


8866 


2) 


MONANOAS OIMLEAWWASILNY JO Ldd LNIOd N NIVLdO OD 


(T666°ZLNOL) ALIUM 
(N°L=I°(I)X) (€666‘ZLNOI) BLIUM 
(SONANOTS OIMLEAWWASILNV ‘IVNIDINO HZE) LYWYOd 
(6866°7LNOL) ALIUM 
ANNILNOO 
LEK #22 
LdZON‘L=I 08 OG 
“0 = (LdZON) xX 
"O° = (PIE 
FNONILNOOS 
(I)X- = (LONI)x 
i = 24 = (ant 
6.0.) = i Fan CTX 
CON‘Z=I OL Od 


) 


6866 


08 


OL 


2) 


AONGNOAS LNIOd N OIMLAWWASILNV ALVA OD 


(T666°7CLNOL) ALIUM 

(LdZON* L=I‘(I)X) (€666°ZLNOL) aALIUM 
(WASLAUI JO LAdLNO HLL) LYWYOd 
(0666‘ZLNOL) ALIUM 

(A ‘N *X)WASLAI TIvoO 


LAd LNIOd ZON WOUd AONATNOAS IVNIDOIUO NIVLdO OL WASLAI ANILNoOugnsS 


(T666‘°7LNOL) ALIUM 
(LdZON* L=I°(I)X) (€666‘ZLNOL) aALIUM 
(WASLAU4 JO LAdLNO HLL) LYWYOd 
(L666‘ZLNOL) ALIUM 
(A ‘N *X)WASLAA TIvo 
ANNILNOO 
CEVA J VRP 
LdZON‘ L=I 09 OG 


Laid LNIOd CON WOUd LAd NIVLEO OL WASLaa ANT LNOWGNS 


( HL/ HL) LVWuOd 

(T666°7LNOL) ALIUM 

(S°€LaS * HL) LYWuOd 

(CaN L=1* (1) x) (€666° CLNOL) ALIYM 

(AONANOAS OIUYLAWWAS dO Ldd LNIOd HZE ‘PI ¢ HL) LVWuOd 
N (7666‘ZLNOL) ALIUM 

(N *X)LSVa TIVO 


FFT Programs 


L666 


09 


HONANOAS LNIOd N dO Ladd ANUL ALNdWOD O 


xipueddy 


SANILNOWMNS Lad LSAL OL SHIONANOAS NASOHD AIWOGNVY 


(7T666°7ZLNOL) ALIYM 

(N° L=I‘(I)X) (€666‘ZLNOL) ALIUM 
(HONANOAS OIULAWWAS IVNIDIYNO H8Z) LYWYOd 
(S666‘°ZLNOL) ALIUM 


ANNILNOOD 

(LRA TS LER 
LdZONSL=I OS OG 
S°O = (0) TaN =: (Lacon yx 
S°O.- £6 )28n e011 )x 
ANNILNOO 

(I)X = (LONI)X 

I - CaN = Lanz 

G°O>>, (0) ER © t2)2 


CON‘ Z=1I OF OG 


S666 


OS 


Ov 


o 


N @ZIS dO AONANOAS IVOINLEWWAS ALVAYNOD O 


L + VON = LdVON 


V/N = VON 
L + CON = LdZON 
c/N.= CON 


cf + N = ZdN 
(7666°7LNOL) ALIUM 
(SHONANOAS WOGNVY HLL ‘SI ‘=N ONILSHL HLL) LVWuOd 
N (9666‘ZLNOI) ALIUM 
OL OL OD 
(960% OL @ AONVY AHL NI Z@ JO UBMOd V LON SI N HSP) LYWYNOd 
(L666°SLLNOI) ALIUM 
ANNILNOOS 
O0€ OL OD (LSELI OR’ NN): JI 
L¥eg =. LSSLT 
cTL°L=I O02 Od 
dOLS (0°OH'N) JI 
(VI) LVWHOd 
N (8666‘QNI) avay 
(=(71)(9600'AT'N'AT'7)AZIS Ldd HOE) LYWYOa 
(6666°LLNOL) ALTIUM 


2) 


9666 
O€ 


L666 


0c 


8666 


6666 
OL 


2) 
Lig WOd AZIS SISAIVNV NI dvgau O 
e) 


(Z)HOVWLI TLNOL 
(7)HOVWLI = LLNOL 
(L)HOVWLI = ANI 


*(@LNOI) LINA LAdLno 


GuaVGNVLS AHL NO NALLIYM SI LAdLNO TIV :LnaLno 


“UaMSNV GHL NI SddAL YaSN AHL GV (LLNOI) IVNIWUAL 
Yas AHL NO NALLIYM SI NOILSHNO V - SI LVHL 


HATLOVYALNI-WaASN SI WVYSOUd SIHL OL LNdNI :LNdNI 
Sdqd00 daOIAdC O0/I ANIAYA 


(S607)A (8606) xX NOISNSWIa 


9607 => N =>Z 
(@ dO YaMOd V 4d LSNW N) HLONAT Ldd AHL SIN 
SdILYadOdd IVIOddS HLIM SHONANOAS Yo 


VL6L0 “AXSUGC MAN ‘TIIH AVUMAW ‘SHINOLVUOSWVI T1494 


UANIGvVa ud > dOHLNV 
SANILNOWYGNS L4d YOd WVUDOUd LSAL :WvuYSOud NIVW 


> LOdGNI 


O Of) 0 O-O 0-0 0 


ODO UODUD ODO UUD 


1.3-14 


FFTSYM 


Section 1.3 


ana 
OL OL OD 

(LHL) LVWHOd €L66 
(€L66°ZLNOL) ALIUM 

5 

a5Vd MAN V NIOAA O 


“ (7666‘7LNOL) ALIUM 

(LdVONSL=I‘(I)X) (€666‘7LNOI) ALIYM 
(HOVLAI dO LAdLNO HLL) LVWHOA VPL66 

(pL66°ZLNOL) ALIYM 

(A *‘N *X)HOVLAI TIVO 


LId LNIOd P/N WOU FAONANOAS ‘IVNISIYO NIVLEO OL HOVLAI ANILNOWMNS ASN O 


(7T666°ZLNOL) ALIUM 

(VONSL=I‘(I)X) (€666°2LNOI) ALIUM 
(HOVLd4 JO LOdLNO HLL) LVYWHOdA SL66 

(SL66‘Z7LNOL) ALIUM 

(AN. “XAIBOVLAG WIVd 


Lid LNIOd VON WOU LAG NIVLEO OL HOVLAdT ANILNOWANS ASN O 


(7666°7LNOL) ALIUM 
(NS L=I‘°(I)X) (€666°7LNOL) ALIUM 

(SJONANOAS OLULAWWASILNVY ‘OINOWYVH GdO 'IVNIDIYO H9P) LVWHOA 9166 
(9L66°ZLNOL) ALIUM 
(N *X)LSS4 TIVO 

*y) 

AONANOAS AWIL OL ANOVA WHOASNVUL O 

o 
(Z666°ZLNOL) ALIUM 
(TAN°L=I‘(I)X) (€666°7LNOL) ALIYM 
(SONSNOAS H6 * 
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Mixed Radix Fast Fourier Transforms 


Richard C. Singleton 


SRI International 
Menlo Park, CA 94025 


1. Purpose 


Mixed radix fast Fourier transform algorithms are presented, for both real and complex input 
sequences. These algorithms also perform multi-variate Fourier transforms. In addition, the program 
used to test the Fourier transform subroutine includes a fast in-place sort subroutine (with tag array) 
and a short bi-variate normal random number subroutine. 


2. Method 


The subroutine FFT evaluates the equation 
NI — ; 2 nk 
X(k) = ¥& x (ne x 

n=0 
where x(n) and X(k) are complex, and k = 0,1,...,N—1. This subroutine also computes the 1/N 
scaled inverse Fourier transform. The subroutine REALT, when called following FFT, completes the 
final transform step for a real sequence x(n) of length 2N. Earlier versions of these algorithms were 
published in 1969 [1], and the methods used were described then in detail. This paper has also been 


reprinted [2], without program listings. 


3. Usage 
3.1 Single-Variate Transform of Complex Input Data 
CALL FFT (A,B,1,N,1,—1) (transform, no scaling) 
or 
CALL FFT (A,B,1,N,1,1) (inverse, scaled by 1/N) 


where 


A Real array dimensioned to size N. On input, A contains the N real components of the data 
to be transformed. On output, the array A contains the Fourier cosine coefficients. The 
roles reverse for the inverse transform. 


B Real array dimensioned to size N. On input, B contains the N real components of the data 
to be transformed. On output, the array B contains the Fourier sine coefficients. The roles 
reverse for the inverse transform. 


N _ Size of input sequence, N. 
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3.2 Multi-variate Transform of Complex Input Data (Tri-variate Example) 


CALL FFT(A,B,N2#N3,N1,1,—1) 
CALL FFT(A,B,N3,N2,N1,—1) 
CALL FFT(A,B,1,N3,N1*N2,—1) 


where the input parameters are the same as in 3.1 above, except that arrays A and B are dimensioned 
(N1,N2,N3). The three calls to FFT can be arranged in any order. The inverse is computed by repeat- 
ing the above calls, but with the sign of the sixth parameter changed to plus. 


3.3 Single-variate Transform of 2*N Real Values, Stored Alternately A(1),B(1),...,A(N),B(N) 


CALL FFT(A,B,1,N,1,—1) 
CALL REALS(A,B,N,—1) 


Or 


CALL FFT(A,B,1,N,1,—1) 
CALL REALT(A,B,1,N,1,—1) 


where the Fourier cosine coefficients are found in A(1),A(2),....A(N),A(N+1) and the Fourier sine 
coefficients are found in B(1),B(2),...,B(N),B(N+1). The inverse is computed by changing the sign of 
the sixth parameter to plus, and reversing the order of the two calls; order is important. Scaling of the 
Fourier coefficients is the same as if FFT had been used to transform 2N real values in A, with B set to 
zero. As indicated above, either REALS or REALT can be used for a single-variate real transform; 
REALS is about ten percent faster than REALT. 


3.4 Bi-variate Transform of Real Input Data, N=N1*N2 


CALL FFT(A,B,N2,N1,1,—1) 

CALL REALT(A,B,N2,N1,1,—1) 

CALL FFT (A(N+1),B(N+1),1,N2,1,—1) 
CALL FFT(A,B,1,N2,N1,—1) 


where the real values are stored alternately in A and B. In this example, arrays A and B are dimen- 
sioned ((N1+1)*N2), and a 2*N1 by N2 transform is done. The call on FFT for the real dimension 
must be first, followed immediately by the corresponding call on REALT. (The folding frequency 
cosine coefficients are then in A(N+1),A(N+2),...,A(N+N2), and the corresponding locations in 
array B are set to zero.) The remaining calls may be in any order. If the sequence of calls is reversed 
and the sign of the sixth parameter is changed to plus, an inverse transform is done. 


3.5 Multi-variate Transform of Real Input Data (Tri-variate Example N=N1*N2*N3) 


CALL FFT(A,B,N3*N2,N1,1,—1) 

CALL REALT(A,B,N3*N2,N1,1,—1) 
CALL FFT(A(N+1),B(N+1),N3,N2,1,—1) 
CALL FFT(A(N +1),B(N +1),1,N3,N2,—1) 
CALL FFT(A,B,N3,N2,N1,—1) 

CALL FFT(A,B,1,N3,N2*N1,—1) 


where the real values are stored alternately in A and B. In this example, arrays A and B are dimen- 
sioned ((N1+1)*N2*N3). The order of the first two calls is fixed, but the remaining calls may be per- 
muted in any order. If the sequence of calls is reversed and the sign of the sixth parameter is changed 
to plus, an inverse transform is done. Adding one more dimension adds two more calls on FFT, one 
for the main array and another for the folding frequency values. There is no limit on the number of 
dimensions. 


The "real value" dimension need not be the column dimension, but the calls on FFT and REALT for 
the real value dimension must come first. 
3.6 Interleaved Storage of Real and Imaginary Data Values 


In any of the above examples, the array B can be replaced by A(2), and the magnitude of the sixth 
parameter set to 2. In this option, A(1),A(3),A(5)... hold the real values and A(2),A(4),A(6),... hold 
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the imaginary values. The values of the third, fourth, and fifth parameters are unchanged. This 
method of calling is particularly useful when transforming real data, in which case the first dimension is 
the real dimension...as in the examples in 3.4 and 3.5 above. 


3.7 Explanation of the Role of Parameters NSEG, N, NSPN and ISN 


FFT (A,B, NSEG,N,NSPN,ISN) 
REALT(A,B,NSEG,N,NSPN,ISN) 


ISN The sign of ISN determines the transform direction, negative for forward and positive for 
inverse. The magnitude of ISN determines the indexing increment for arrays A and B. 


N The dimension of the current variable. 


NSPN The spacing of consecutive data values while indexing the current variable, in units determined 
by the magnitude of ISN. 


NSEG NSEG#*N«#NSPN is the total number of complex data values. 


Users familiar with the original 1969 version should note the changed meaning of the third and fifth 
parameters; to convert a program calling the earlier version of FFT, the user should replace NSPAN by 
NSPAN/N and should replace NTOT by NTOT/NSPAN. This change was made to eliminate two pos- 
sible error conditions. 


3.8 Error Conditions 


The signs of N, NSPN, and NSEG are ignored, and any nonzero value can be valid. Any nonzero 
value of ISN can be valid, although it is not anticipated that magnitudes other than one or two will 
often be used. Thus the following error condition is recognized in FFT and REALT: 


NSEG*Nx«NSPN#ISN = 0 


This condition results in an error message and no transform. The corresponding error condition for 
REALS is N#ISN = 0. 


If N has more than 15 "factors", FFT terminates with an error message. The smallest number with 
16 "factors" is 12,754,584, so this error condition should not occur very often. (See [1] or [2].) 


4. Tests 


The subroutines included in this section have had extensive use in their earlier versions, and the 
modifications for inclusion in this publication have been held to a minimum. To enable the user to 
verify that the subroutines are working correctly on his system, a test program with sample output is 
included here. 


The tests used here first do a transform-inverse pair on N = 32 real values, using FFT, REALS and 
REALT. The input value, the Fourier coefficient values, then the inverse result are printed; the final 
inverse values are identical with the input values. The sorted error values for arrays A and B are also 
listed. 


The same sequence of N = 32 real values is then stored in array A, with array B set to zero, and a 
transform-inverse pair is done using FFT alone. The Fourier coefficients and final inverse values are 
printed; the Fourier coefficient values are seen to be the same as those obtained in the previous test. 
The first two tests are done using pseudo-random numbers on the unit interval. For the remaining 
tests, independent bi-variate random normal deviates with zero mean and unit standard deviation are 
used. The advantage of this approach is that the input data has an RMS value of one; the RMS error 
value for the final transform-inverse result then has a simple reference. 


The next set of tests runs through a variety of test examples, with only the RMS error values for 
the transform-inverse pair printed. These examples are designed to test all branches of the subroutine. 


The final test lists the Fourier coefficients for an 8 by 9 real transform, computed both by FFT alone 
(with zeros in the imaginary array) and by FFT and REALT. The results are identical, and illustrate 
the method used in REALT to store the folding frequency coefficients. 
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Table 1 


TEST OF FFT, REALS AND REALT FOR REAL VALUES 


REAL INPUT SEQUENCE --- 


J A(J) B(J) A(J+1) B(J+1) 

1 -22925607E+00 - 76687502E+00 -68317685E+00 -50919111E+00 
3 -87455959E+00 -64464100E+00 -84746840E+00 .35396343E+00 
5 -39889159EF+00 -45709421E+00 .23630936E+00 .13318189E+00 
7 -16605222EF+00 .22602680E+00 -66245903E+00 -25021174E+00 
9 .61769668E+00 .26246527E+00 -51266762E+00 -93920734E+00 
11 .62402816E+00 .42238195E+00 -93970599E+00 -28206823E+00 
13 .46921754E+00 -54879178E-01 -51983086E+00 -39682690E+00 
45 -11315656E+00 -60751725E+00 .70150672E+00 -88705479E+00 

FOURIER COSINE AND SINE COEFFICIENTS --- 

J A(J) B(J) A(J+1) B(J+1) 

1 -15789569E+02 0. .13026034E+01 -19871480E+00 
3 -11936771E+01 -.28999118F+01 .40612714E-01 .72163937E+00 
5 -.46179909EF+00 -13081519E+01 -70122852E-01 -15931004E-01 
7 -.27579355E+00 -10351044E+01 .46209660E+00 -11709983E-01 
9 -.16102664E+01 .30982473E+00 .14930917E+01 -87044475E+00 
TT -21015246E+00 -.18331043E+00 -93344462F+00 -55190813E-01 
43 -33632980E+00 -.10901590E+01 -12900518E+01 -18219048E+01 
15 -.12126616E+01 .75918926E+00 -12663723E+01 -10652233E+00 
sa -14023971E+01 0. 

TRANSFORM-INVERSE RESULT --- 

5 A(J) B(J) A(J+1) B(J+1) 

1 -22925607E+00 . 76687502E+00 -68317685E+00 -50919111E+00 
3 -87455959F+00 -64464100E+00 .84746840E+00 -35396343E+00 
5 .39889159EF+00 -45709421E+00 .23630936E+00 -13318189E+00 
ei -16605222E+00 .22602680E+00 -66245903E+00 .25021174E+00 
9 .61769668E+00 .26246527E+00 .51266762E+00 -93920734E+00 
7 -62402816E+00 .42238195E+00 -93970599E+00 .28206823E+00 
43 -46921754E+00 -54879178E-01 -51983086E+00 -39682690E+00 
15 -11315656E+00 -60751725E+00 .70150672E+00 -88705479E+00 


SORTED ERROR VALUES --- 


Aj A(J) J B(J) 
16 - .36379788E-11 4 -.36379788E-11 

6 -.36379788E-11 6 -.36379788E-11 
43 -.18189894E-11 42 -.36379788E-11 
12 os 7 -.18189894E-11 
10 0. 1 > 
14 0. 13 e-. 

9 0. 14 0. 

8 0. +7 0. 
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(Continued) 

9 

5 

: 3 
£22°737368E=42 2 
.18189894E-11 8 
.18189894E-11 15 
.36379788E-11 10 
.36379788E-11 16 


ao MR come Ya eT a) 


-18189894E-11 
.18189894E-11 
. 36379788E-11 
-54569682E-11 


TEST OF FFT FOR REAL VALUES 


FOURIER COSINE AND SINE COEFFICIENTS --- 


A(J) 


. 15789569E+02 
. 119367 FI +01 


-46179909E+00 
.2/7579355E+00 
.16102664E+01 


.21015246E+00 
.33632980E+00 


-12126616E+01 


-14023971E+01 


-12126616E+01 


.33632980E+00 
-21015246E+00 


.16102664E+01 
-275793556700 
-46179909E+00 


~ 1193677275401 


.28999118E+01 
.13081519E+01 
-10351044E+01 


B(J) 


~o0962473E+00. -~2 


.18331043E+00 - 
1090 TSS90E+07 =. 


~TS97B926E+00 -. 


~7T591S926E+00 —. 
«109015908101. = 


.18331043E+00 -. 


TRANSFORM-INVERSE RESULT 


A(J) 


-22925607E+00 
.68317685E+00 
.87455959E+00 
.84746840E+00 
.39889159E+00 
.23630936E+00 
-16605222E+00 
.66245903E+00 
.61769668E+00 
-51266762E+00 
.62402816E+00 
. 939705995400 
-46921754E+00 
-51983086E+00 
-11315656E+00 
.70150672E+00 


.30982473E+00 
.10351044E+01 
-13081T519E+01 
.28999118E+01 


.40946464E-12 
.41774510E-13 
.42896152E-12 
saatsl s68E-12 
.40946464E-12 
.86772019E-12 
.98682366E-12 


~45282706E-13 
<aea 1 96228 =1-1 
.59949178E-12 
:22937268E-12 
i 1743 F867E-11 
« P32Z24679E-11 
9320 7631E-12 


B(J) 


A(J+1) 


.13026034E+01 
.40612714E-01 
~70122852E-01 


12663723E+01 


.46209660E+00 
«70722852E-01 


B(J+1) 


.19871480E+00 
c4216395/E+00 
- 15931004E-01 


.46209660E+00 .11709983E-01 
14930917E+01 .87044475E+00 
.93344462E+00 .55190813E-01 
12900518E+01 .18219048E+01 


-10652233E+00 


. 12663723E+01 . 10652233E+00 
12900518E+01 .18219048E+01 
.93344462E+00 -595190613E-01 
1493091 7E+01 .87044475E+00 


- 11 709983E-01 
-15931004E-01 


.40612714E-01 .72163937E+00 
.13026034E+01 .19871480E+00 
A(J+1) B(J+1) 
.76687502E+00 .45474735E-12 
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; /48E+=11 
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=193E- 10 
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-850E-11 
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.842E-11 


2I2ZE=t0 
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.853E-11 
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. 849E-11 
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.834E-11 
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.875E-11 
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.886E-11 
. 886E-11 
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<9 7E=11 
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<T9TE-11 
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.202E-10 
.833E-11 
(832E-11 
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.805E-11 
.804E-11 


.283E-10 
-821E=—T1 
.820E-11 


.284E-10 
.833E-1T1 
.832E-11 


- 391E-10 
s825E=11 
.824E-11 


.288E-10 
.833E-11 
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.226E-19 
. 7695-11 
. 764E-11 


.221E-10 
.847E-11 
.843E-11 


.402E-10 
.862E-11 
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RMS ERROR 
N A ARRAY B ARRAY 
FFT 1080 .162E-10 . 153E-10 
FFT 1080 <tt9E-10 ‘ 
REALT 1080 218-71  9F2E=11 
REALT 1080 ee fee 
8 BY 9 REAL TRANSFORM --- FFT VS. 
A(J) C(J) 
«185091828402 0. -.18509182E+02 
.63947267E+01 .97863880E+01 -.63947267E+01 
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.78263712E+01 
.28572332E+01 
.10921541E+02 
.28572332E+01 
.78263712E+01 
.63947267E+01 
.46916612E+01 
.70044347E+01 
.25491851E+01 
.57953108E+00 
.16600946E+01 
.56098061E+01 
.73299048E+01 
.18873938E+01 
.89631614E+01 
.30901046E+01 
.21382018E+01 
.67686785E+01 
.50517953E+01 
.16673708E+01 
.16322342E+02 
.38034461E+01 
.52371225E+01 
.56868554E+01 
.25913244E+01 
.25446027E+01 
.89007166E+01 
.34296333E+01 
.13925042E+01 
.35248262E+01 
.72919622E+00 
.42411487E+01 
.38718108E+01 
.32136390E+01 
.54606791E+00 
.61526515E+01 
.56624794E+01 
.15985812E+01 
.72919622E+00 
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.38034461E+01 
.16322342E+02 
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.50517953E+01 
.67686785E+01 
.21382018E+01 
.30901046E+01 
.46916612E+01 
.18873938E+01 
.73299048E+01 
.56098061E+01 
.16600946E+01 
.57953108E+00 
.25491851E+01 
.70044347E+01 
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-25613030E+01 
-10412241E+00 


.10412241E+00 
.25613030E+01 
.97863880E+01 
.51173453E+01 
.40156402E+01 
.37405183E+01 
.32320996E+01 
.23477400E+01 
.48183139E+01 
.85799253E+01 
.22840714E+01 
.30857869E+01 
.92397343E+01 
.49035087E+01 
.76038599E+01 
.67468637E+01 
.28199019E+01 
.10742899E+02 
.13483020E+02 
.98372729E+01 
.61338098E+01 
.12416175E+02 
.11353010E+01 
.30849262E+01 
.39694987E+01 
.41322696E+01 
.23698139E+01 
.24843916E+01 
.11135565E+02 
.31550578E+01 
.21105050E+01 
.22407029E+01 
.11118265E+02 
.14726845E+02 
.91010428E+01 
.24843916E+01 
.91010428E+01 
.14726845E+02 
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.39694987E+01 
.30849262E+01 
.11353010E+01 
.12416175E+02 
.61338098E+01 
.30857869E+01 
.13483020E+02 
.10742899E+02 
.28199019E+01 
.67468637E+01 
.76038599E+01 
.49035087E+01 
.92397343E+01 
.51173453E+01 
.22840714E+01 
.85799253E+01 
.48183139E+01 
.23477400E+01 
.32320996E+01 
.37405183E+01 
.40156402E+01 
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.70044347E+01 
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.34296333E+01 
.89631614E+01 
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Optimized Mass Storage FFT Program 


Donald Fraser 


CSIRO, Division of Computing Research 
Canberra City, ACT 2601, Australia 


1. Purpose 


The program computes the discrete Fourier transform (DFT) and inverse discrete Fourier transform 
(IDFT), in one or more dimensions, of a (large) array in mass store. 


2. Method 


The program is an implementation of the optimal sorting algorithm of the author [1] which allows a 
base 2 version of the Cooley-Tukey FFT algorithm [2,3,4] efficient access to a mass store array. 
Optimal sorting for the mass storage FFT has been determined independently by DeLotto and Dotti 
[5,6], but in the author’s version the emphasis is on ‘in-place’ array modification. This results in 
slightly higher mass store I/O than the minimum, but requires no additional mass store working space. 
The method is also discussed in [7] and is a logical extension of the work of Singleton [8] and Brenner 


[9]. 


The program computes in-place the discrete Fourier transform of a one-dimensional or a multidi- 
mensional array. In the one-dimensional case the transform is defined by 
N—| + j 25 nk 
X(k) =SCAL ¥& x(n)e_ *% , k=0,1,...,.N—1 (1) 
n=0 
where SCAL is an arbitrary scaling factor, and j = V—1, the sign of the exponent being either minus 
(DFT) or plus (IDFT). 


The definition Eq. (1) is easily generalized to cover more than one dimension; for example the 
two-dimensional case is given by 


N,-1 Ny-1 = jan “HL 4 ad 
X(ky,k2) =SCAL Y} DY x(nj,,n)e . ; (2) 
n,=0 n,=0 


for ky = 0,1,...,.N,—1 and k, a 0,1,...,N.—1 


The elements x(n) or x(m,,n,) in Eq. (1) or Eq. (2) are the initial complex data in a mass store 
array. These are replaced by the elements X(k) or X(k,,k>) as the final complex data. 


Complex data is transformed by an in-place, base-2 algorithm using post-computation bit-reversal to 
sort the array [3]. The computation is handled by a modified, in-core FFT routine which does a 
sequence of partial transforms of the mass store array (the method is discussed in more detail in section 
5). The sorting algorithm [1] calculates the most efficient way to access the mass store array for these 
computations. Finally the sorting algorithm is used to carry out an overall bit-reversed permutation of 
the array, again with as few accesses as possible (I/O efficiency is discussed in section 4). 


Computation and sorting occur ‘in-place’, through a combination of ‘virtual’ permutations, where 
mass store blocks are accessed according to an indexing algorithm but are left physically unpermuted, 
and symmetric permutations, which interchange blocks according to a generalized index bit-reversal. In 
[1] it is shown that any unsymmetric permutation can be formed from two suitable symmetric 
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permutations, each of which can be done in-place. 


Multidimensional transforms are achieved automatically by making use of the indexing structure of 
the FFT algorithm itself. No change to the order of access of elements is necessary, so that the full 
advantage of the sorting algorithm and program simplicity is maintained. 


For transforms in which the initial or final data is real, the usual time saving algorithm [4] is avail- 
able to unscramble a half length complex transform of packed real data (or vice-versa). This requires 
an extra accessing and computing pass through the mass store array, but still results in a saving of 
nearly half the computation and I/O time. In this method, it is easy to allow a choice in the degree of 
redundancy in the final, complex result. The array may be expanded to full redundancy (twice the phy- 
sical length of the original packed real array), or to partial redundancy, or maintained the same physical 
length by elimination of all redundancy. 


Finally, we must define a number of terms used in the discussion. ‘Core store’ is used to describe a 
region where the elements of an array are equally accessible at random. ‘Mass store’ implies a region 
where elements are grouped into ‘blocks’ or ‘records’ which must be accessed as units, but which units 
are accessible efficiently at random. A ‘pass’ is an array operation which leaves the array elements in- 
place ready for another array operation. Thus, an ‘I/O pass’ reads and writes back once all the blocks 
of an array (this is a logical definition — sometimes in practice not all blocks need be physically 
accessed, or some may be accessed more than once). 


3. Program Description 
3.1 Usage 


Figure 1 shows the structure of the internal subroutine calling network. 


5 3 USER 
MFPAR}= ~~ —- rer 
HELPER os 
= sas | 
ae 3 
Se 


[MF SUM| 


MFCOMP } ™ UNSCRAMBLE 
a 


- CORE FFT Ts 
sae alge MFRLOD 
— ~>] ME INDX 
ee LOAD 
MELOAD VIRTUAL PERM 


MFREAD,MFWRIT 


MASS STORE 1/0 


Fig. 1 Internal subroutine calling network. (Solid and dashed lines distinguish roughly 
between ‘main path’ and ‘reference’ calls). 


The program consists of a set of subroutines written in ANSI FORTRAN, only two of which are called 
by a user program. If the data are always complex the user program calls subroutine CMFFT. If the 
input data or the results are real, the user program may call the faster subroutine RMFFT. Either sub- 
routine replaces a one-dimensional or a multidimensional array in mass store by its discrete Fourier 
transform, as defined in section 2. 


(If the elements of the mass store array are considered to be singly indexed by i=0,1,2,..., then the 
indices of the definitions map into i=n in the one-dimensional case or i=n,+n,:N, in the two- 
dimensional case, resultant transposition giving i=k,+k,-N,. In general, i=n,+n,-N,+n3:N>‘N}+... 
Remember also that for FORTRAN, indices are increased by 1.) 
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The mass store file is assumed to exist and to have been previously defined to the FORTRAN sys- 
tem and opened for random access by the user. The file is accessed through two system-dependent 
subroutines MFREAD and MFWRIT (see section 7). All other subroutines are system-independent. 


The user has freedom to specify total array size, mass store block or record size, core store working 
space size and the dimensioning of the array, except that all sizes are to base 2 and are given by their 
binary exponents. 


3.2 Calling Method 
The complex FFT routine is called by 
CALL CMFFT (MEXA,NDIM,ISGN,IDIR,SCAL,BUFA,IBEX,ICEX) 
while the faster, real-to-complex (or vice-versa) FFT routine is called by 
CALL RMFFT (MEXA,NDIM,ISGN,IDIR,SCAL,BUFA,IBEX,ICEX,IPAK) 


3.3 Description of Parameters 


MEXA Integer array of size NDIM (see below). MEXA consists of a list of dimension size binary 
exponents, defining the dimensioning of the mass store array. For example, a one- 
dimensional array has a size of 2**MEXA(1) elements. A two-dimensional array has 
2** MEXA(2) sets of 2**MEXA(1) adjacent elements each (N of definition Eq. (1) or N, 
and N, of definition Eq. (2) are 2** MEXA(1) and 2** MEXA(2) in initial order). 


Notes: 


a) The FFT routines actively modify the MEXA list, if necessary, leaving it in the order 
corresponding to the final array dimensioning. In general, its order is reversed by the 
FFT, except in the special case with routine CMFFT discussed under parameter IDIR 
below. The MEXA list should therefore be linked to a unique mass store array so that it 
always indicates the current dimensioning of that array. 


b) The MEXA list of exponents always refers to the dimensioning of a full array, as required 
by routine CMFFT or by routine RMFFT with IPAK=+1 (see below). The complex 
result of routine RMFFT operating on real data is truncated when IPAK=0 or —1, so that 
the MEXA list elements in these cases may refer to a virtual array length. 


c) The first element of the list always gives the size 2** MEXA(1) of the set or sets of adja- 
cent elements in the first dimension of the mass store array. If data are of type complex, 
this is the number of complex elements in each set while if data are of type real (e.g. 
before calling routine RMFFT in direction real-to-complex), this is the number of real 
elements. 


d) Mass store array dimensioning is independent of I/O block transfer size and core working 
space size. 


NDIM Number of dimensions in mass store array (size of MEXA list). 


Range 1 < NDIM < 4 (RMFFT), to increase, see section 5. 
or 1 < NDIM < M (CMFFT). 


(M) Not a call parameter, but defined here for convenience. M = Sum to NDIM of MEXA list, 
giving total mass store array size = 2** M elements (number of complex elements for routine 
CMFFT, real or complex elements for routine RMFFT, but see under IPAK packing parame- 
ter below). 


ISGN The sign of ISGN is the sign of the complex exponent of the transform definition (e.g. equa- 
tion 1 or 2). A positive or negative sign results in the inverse transform of the other, but see 
also below. 


IDIR Transform direction (RMFFT), reversal (CMFFT) parameter. Routine RMFFT converts 
packed real data to complex data (or vice-versa) during transformation, so that IDIR is 
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SCAL 


BUFA 


IBEX 


ICEX 


IPAK 


needed to determine the direction, independently of ISGN, thus: 


IDIR=—1  Real-to-complex (RMFFT), Dimension reversal 
IDIR=+1 Complex-to-real (RMFFT), Dimension reversal. 


(Note that it is usually most convenient to use the same variable for both ISGN and IDIR so 
that a single negation results in transform inversion). 


Routine CMFFT does not need a direction parameter, other than ISGN, and IDIR is used in 
this case to set dimension reversal (transposition in two dimensions) or not, as required, 
thus: 


IDIR not zero, Dimension reversal (more efficient I/O) 
IDIR=0 DO NOT USE (RMFFT), Suppress reversal (CMFFT). 


Arbitrary type real scale factor of Eqs. (1) or (2). If SCAL=1.0 computation is fastest as no 
scaling occurs. : 


Array in core to be used as workspace by FFT routines. Note that internal FFT subroutines 
assume the following: 


a) BUFA is either type real or complex, to suit local needs. 


b) BUFA is given the trivial dimension BUFA(1) internally, since its actual size is known 
only as the exponent ICEX (see below). 


c) Type complex data is assumed to . be stored in the sequence 
real/imaginary/real/imaginary/.. in core store and mass store. 


Some of these points may upset some FORTRAN compilers. 


Mass store I/O block transfer size binary exponent. Block or record size = 2** IBEX real ele- 
ments. 


IBEX < ICEX—2 (RMFFT), 

IBEX < ICEX—1 (CMFFT). 

Core store working space size binary exponent. Dimension of BUFA = 2**ICEX real ele- 
ments. 


Limits are IBEX+2 < ICEX < M (RMFFT), 
or IBEX+1 < ICEX < M+1 (CMFFT). 


Packing parameter (routine RMFFT) only. 
Determines the degree of redundancy (discussed in more detail in section 5) desired in the 
complex result after a real-to-complex transform, thus: 


Limits are 2 < 
l< 


B 
or B 


IPAK=+1 ‘gives a-fully redundant complex result. The final mass store array is exactly 
twice the physical length of the initial real array, having the same number, 
2*x* M, of complex elements as initial real elements. This is the same result that 
is obtained when calling CMFFT with the initial data occupying the real part of a 
complex array, zero imaginary. 


IPAK=0 gives a partly redundant complex result, slightly longer than the initial real 
array. In this case there are 2** (M—1) + 2** (M—MEXA(NDIM)) complex 
elements in the final array, the MEXA list in the order after the transform 
(increased to an integral number of mass store blocks, if necessary). This is 
probably the most useful packing. 


IPAK=—1__ gives a result containing no redundancy, and having exactly the same physical 
length as the initial 2** M real elements, or 2** (M—1) complex elements. This 
is achieved by squeezing together those parts of the array having internal redun- 
dancy. The exact algorithm is discussed in section 5. No information is lost 
and, for one or two dimensions, only a little sorting is needed to access 
unsqueezed information. For example, in one dimension, the real value at the 
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Nyquist frequency becomes the imaginary part of the zero frequency element, 
which also must be real. 


Notes: 


a) For IPAK=+1 or 0 the mass store array file must be extendable. IPAK=—1 has the advan- 
tage that the mass store array file remains a fixed length, but has the disadvantage that some 
user effort is needed to access complex data correctly. 


b) In the complex-to-real direction, IPAK=+1 or 0 are equivalent, since only part of a fully 
redundant complex array is accessed by RMFFT in this direction. But IPAK=—1 must be 
used in both directions to correctly handle the squeezed complex array. 


3.4 Helper Routine 


To help the user set up the arguments of section 3.3 and to determine the mass store file and block 
sizes, a helper routine MFPAR is included. Use of the routine is not essential, but is recommended. 
The routine is called by: 


IERR = MFPAR (IRMF,ICOMP) 


with IRMF=—1 if mass store data is currently packed real or +1 if data is currently complex when using 
routine RMFFT. When using routine CMFFT, IRMF=0. If ICOMP=0 the argument exponents 
MEXA(), IBEX and ICEX are defined by the user while if ICOMP is not zero the exponents are to be 
computed by MFPAR. 


Three COMMON blocks are used to transmit other data, thus: 


COMMON/MFARG/ MEXA(4),NDIM,ISGN,IDIR,SCAL, IBEX,ICEX IPAK 
COMMON/MFVAL/ DIMA(4),TDM1,RDM1,FBLK,TBLK,RBLK,RCOR,SIZE 
COMMON/MFINT/ NDMA(4) ,NTD1,NRD1,NFBK,NTBK,NRBK,NRCR,NSZE 


MFARG holds a four element MEXA() list (for up to four dimensions) followed by the other 
mass store FFT call arguments (except BUFA). NDIM, IBEX, ICEX and IPAK must be present 
(unless ICOMP not zero), while ISGN, IDIR and SCAL are ignored here. 


COMMON blocks MFVAL and MFINT return data computed by routine MFPAR. These include 
the four element arrays DIMA() and NDMA() corresponding element by element to MEXA() but 
containing the actual dimension sizes 2** MEXA(). Similarly RBLK, NRBK and RCOR, NRCR hold 
the sizes 2**IBEX and 2**ICEX. Note that MFINT variables are one to one integer conversions of 
MFVAL variables, which are type real. A local variable FIXMAX, the biggest positive integer in the 
machine, determines whether a conversion is allowed. Any values not converted are set to —1 in 
MFINT and the function returns MFPAR=-—1 to indicate this. IBEX and ICEX are forced to be within 
the limits defined in section 3.3, and MFPAR=+1 if IBEX is forced too small. Otherwise the function 
returns MFPAR=0 normally. 


If the helper subroutine argument ICOMP is not zero, the computation is reversed and MEXA( ) 
exponents are computed from given DIMA( ) real sizes, IBEX and ICEX computed from RBLK and 
RCOR real sizes. Sizes are adjusted to be integral powers of 2, adjusted up or down to the closest 
power on a log scale. NDIM and IPAK must still be given in MFARG. 


The most useful values computed by MFPAR are NTD1, NRD1, NFBK, NTBK and NRBK (in 
MFINT). NRBK is 2** IBEX and is the number of reals in the ‘record’ used for mass store access by 
the FFT. NTBK is the current total number of records of size NRBK, or the current file length. 
NFBK, on the other hand, is the maximum number of records of size NRBK to be expected, including 
any mass store array expansion by routine RMFFT (see IPAK=0 or 1 in section 3.3). NFBK is thus 
useful for defining the maximum file length to the operating system. 


NRD1 is the number of reals in an equivalent ‘record’ of the current first dimension length (‘first’ 
defined by current MEXA(1)). NTD1 is the current number of such equivalent records and these two 
values are useful for logically accessing a multidimensional mass store array by the user (though not so 
useful if NDIM=1). Note that to do this the I/O routines MFREAD and MFWRIT must be able to 
handle correctly ‘records’ different in length from NRBK, otherwise NTBK and NRBK should be used. 
This will be system dependent (see section 7). 
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NSZE, SIZE is the effective total size of of the mass store array (2%* M) and is useful, for example, 
in computing the scale factor SCAL. 


Note that MFPAR should be called just prior to user file access, and with the correct value for 
IRMF, to ensure the computed variables reflect the current state of the mass store array. 


3.5 Example 


Suppose we wish to transform a two-dimensional real array having 512 rows (2x*9) of 256 adjacent 
real elements each (2**8), or a total of 128K real elements (2**17), where K = 1024. Suppose we 
decide to allow a core store working space of 8K real elements (2**13) and to access the mass store 
array in blocks of 128 real elements (2**7) each, these being quite independent of array dimensioning. 
Then, using the helper routine MFPAR (section 3.4) to compute file parameters, the mass store file is 
defined and opened for random access, loadd with data and subroutine RMFFT is called with: 


COMMON/MFARG/ MEXA(4),NDIM,ISGN,IDIR,SCAL, IBEX, ,ICEX ,IPAK 
COMMON/MFVAL/ DIMA(4),TDM1,RDM1,FBLK,TBLK,RBLK,RCOR,SIZE 
COMMON/MFINT/ NDMA(4),NTD1,NRD1,NFBK,NTBK,NRBK,NRCR,NSZE 
C COMMON BLOCKS USED BY HELPER ROUTN MFPAR (NOT ESSENTIAL) 
C 
REAL BUFA(8192) 
COMPLEX CBUFA (4096) 
EQUIVALENCE (BUFA(1 ),CBUFA(1)) 
C WORKING ARRAY IN CORE (EQUIVALENCED FOR USER ACCESS) 
% 
MEXA(1)=8 
MEXA(2)=9 
NDIM=2 
ISGN=-1 
IDIR=-—1 
SCAL=1.0 
IBEX=7 
ICEX=13 
IPAK=1 
C MASS STORE FFT ARGUMENTS INITIALIZED 
C 
IRMF=-1 
ICOMP=0 
IERR=MFPAR (IRMF,ICOMP) 
C HELPER ROUTINE, DATA REAL, RMFFT, EXPONENTS DEFINED, IF(IERR.EQ.O) OK 
C COMPUTES FILE PARAMETERS IN COMMON AREAS (NOT ESSENTIAL, BUT USEFUL) 
C 
(open mass store file here) 
C HERE CAN OPEN MASS STORE FILE, NFBK MAX ‘RECDS’ OF NRBK REALS 
C : 
DO 2 JB=1,NTD1 (or NTBK) 
DO 1 I=1,NRD1 (or NRBK) 
BUFA (I= (Enter a real variable here) 


NO 


CALL MFWRIT (BUFA,NRD1,JB) (or (BUFA,NRBK,JB)) 

C LOAD MASS STORE FILE WITH PACKED REAL DATA (NTD1=512, NRD1=256) 

C (IF MFREAD/MFWRIT REQUIRE FIXED RECD LENGTH, USE NTBK,NRBK INSTEAD) 
C 

CALL RMFFT (MEXA,NDIM,ISGN,IDIR,SCAL,BUFA, IBEX ,ICEX,IPAK) 


resulting in the DFT of Eq. (2), with SCAL=1.0, N,=256 and N,=512 and a negative complex 


exponent. Because the initial data are packed real and because the parameter IPAK=1 is chosen, the 
mass store array will be extended to 128K complex elements, or twice its initial physical size. This 
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array is a fully redundant transform of the original, having 256 rows (2**8) of 512 adjacent complex 
elements (2**9), the dimensions being reversed or transposed. To indicate transposition the MEXA 
list will be reversed (MEXA(1)=9, MEXA(2)=8 after the transform). 


To save unnecessary array extension the last parameter IPAK can be made 0 or —1. If IPAK=0 the 
result is a partially redundant transform, being the first 129 rows (2**MEXA(2)/2+1) of the 
transform, of 512 adjacent complex elements each. The first and last rows each have an internal conju- 
gate symmetry, but other redundancy is deleted. 


If IPAK=—1 the first and last rows above are ‘squeezed’ together, the second half of the last row 
becoming the second half of the first row. In addition, the first and middle rea! elements of the last 
row become the imaginary parts of the first and middle elements of the first row, resulting in an array 
of 128 rows of 512 complex elements, exactly the same physical length as the initial array, so that no 
file extension is necessary (see section 5). 


The complex result is accessed, then an inverse transform is called by: 


IRMF=+1 

IERR=MFPAR (IRMF,ICOMP) 
C HELPER ROUTINE AGAIN, DATA COMPLEX, ROUTINE RMFFT, IF(IERR.EQ.O) OK 
C 


NCNT=NRD1/2 (or NRBK/2) 
C NCNT IS THE NUMBER OF COMPLEX ELEMENTS IN RECORD 
DO 4 JB=1,NTD1 (or NTBK) 


CALL MFREAD (BUFA,NRD1,JB) (or (BUFA,NRBK,JB)) 
DO 3 I=1,NCNT 

3 (access each complex element CBUF(I) here) 

Cc 

4 CALL MFWRIT (BUFA,NRD1,JB) (or BUFA,NRBK,JB)) 

C COMPLEX RESULT READ BY USER (WRITING NEW VALUES IF DESIRED) 

C (IF MFREAD/MFWRIT REQUIRE FIXED RECD LENGTH, USE NTBK,NRBK INSTEAD) 
ISGN=-—ISGN 
IDIR=—IDIR 
SCAL= 1075122 | | 
CALL RMFFT (MEXA,NDIM,ISGN,IDIR,SCAL,BUFA,IBEX,ICEX,IPAK) 


the scale factor SCAL, using SIZE=2.** M computed by MFPAR, is chosen here to normalize the 
result to the same scale as the original data. The result is a real array having 512 rows of 256 adjacent 
real elements each, the MEXA list being restored to its initial order. 


3.6 Addendum — Transposing, Dimension Shifting 


The sorting algorithm used in the mass storage FFT may also be used for changing the order of 
dimensioning of a multidimensional mass store array. (This is similar to, but more general than, 
Eklundh’s method [10]). As an example, and in addition to the FFT routines, a dimension shifting 
routine DMPERM is included for completeness, called thus: 


CALL DMPERM (MEXA,NDIM,NSHFT,IREX,BUFA,IBEX,ICEX) 
where the arguments MEXA, NDIM, BUFA, IBEX, ICEX have the same meaning as in the FFT calls. 
The other parameters are: 7 
NSHFT Dimension shift count. 


NSHFT=0 No shift or change occurs. 
NSHFT=1,2 etc. _ First-to-next dimension, circular NSHFT place shift, Modulo (NDIM). 
NSHFT=-1 Dimension order reversed. 


IREX ‘Element size’ binary exponent (size =2** IREX reals), thus IREX=0 for real array, 
IREX=1 for complex array. 
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Most of the comments concerning the FFT routine parameters apply also to this routine. The mass 
store array is either real or complex (or ‘elements’ of multiple reals), with an MEXA list defining the 
current dimensioning (binary exponents). The MEXA list is actively modified by the routine if neces- 
sary. (Note that to sort arrays of elements smaller in size than type real, such as the type byte elements 
of some systems, it is necessary to alter type statements for BUFA and TEMP variables in all subrou- 
tines called). 


The routine is quite trivial in design, consisting of only twenty statements. It operates by calling the 
generalized mass store bit-reversed sorting routine MFSORT, used internally by the FFT routines, and 
therefore has a similar I/O efficiency to the FFT routines (section 4). 


4. Input/Output Efficiency 


The number of I/O passes (see definitions, section 2) through the mass store array depends on the 
array size (2**M), the core store working space size (2®*ICEX) and the I/O block or record size 
(2* IBEX). In general, the larger the working space and the smaller the block size the better, but 
block size should not be made too small because of other overheads. The FFT computation requires 
I((M—IBEX+1)/(ICEX—IBEX)) passes involving I/O (where J(x) is the smallest integer greater than 
or equal to x). Post computation sorting requires a similar number of passes (without sorting overlap), 
although DeLotto and Dotti [5] mention ((M—ICEX+1)/(ICEX—IBEX)) passes. The reason this is 
not achieved is that in-place sorting requires M—IBEX+1 virtual permutations to leave a physically 
unpermuted array. A change in the algorithm to a smaller number M—ICEX+1 virtual permutations 
requires an extra block-sorting pass of mass store, nullifying the advantage. 


Figure 2 gives the maximum range of block size exponent IBEX necessary to keep the number of 
I/O passes, N, of the mass store array low, given ICEX and M. Solid lines bound the optimum N=4. 
Dashed lines are the boundaries for the next best N=6 passes, and it should not be difficult to operate 
with N=4 or 6 in most cases. When calling the half-length transform by subroutine RMFFT, increase 
diagram M and N by one to obtain the working M and N. 


The hatched line cutting across the figure gives a bound below which all mass store passes access all 
blocks. In the area above the line, /((IBEX—1)/(ICEX—IBEX)) sorting passes are required with an 
additional pass of mass store involving a block reshuffle, in which only some of the blocks are accessed. 
That is, the optimum can approach N=3 passes. Given an I/O system which allows alteration of the 
index key of a block, the block shuffle can be replaced by an index shuffle, giving N—1 passes in the 
upper area. Note also that the diagram is only approximate, as integral rounding effects may increase or 
decrease the number of passes by one at some points. 


Run time depends on two main factors, the computation time of the in-core FFT and the I/O time. 
A total elapsed time can be written approximately 


T = TC Mx 2%* M + TM Nx 2% M 


where TC and TM are unit computation and unit mass store average access and transfer times 
corresponding to each complex element. As an example, consider two very different computer sys- 
tems, a PDP 11/40 system and a CYBER 76 installation. Then we may expect TC=0.5 ms (PDP 11) or 
2 ws (CYBER), and TM=1 ms (PDP 11, IBEX=7) or 0.5 ms (CYBER, IBEX=9) for routine CMFFT. 
Calling subroutine RMFFT with packed real data roughly halves the time, so that a 256x256 real array 
can be transformed in about 5 minutes (PDP 11) or 1 second (CYBER) CPU time and 8 minutes (PDP 
11) or 1 minute (CYBER) elapsed time. But in machines such as the CYBER, where computing speed 
is very great compared to I/O latency, the routines are used to great advantage for transforming arrays 
in LCM (extended memory) in place of mass store, using high speed block-copy operations between 
this and main memory (see sections 6 and 7). Whether an array is multidimensional or not makes little 
difference to efficiency. 


Both TC and TM are block size dependent, there being fixed overheads per block. Both therefore 
can be written 


TC = TCO + TCA/2*% (IBEX—1) and TM = TMO + TMA/2«* (IBEX—1) 
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IBEX max —_— 


Fig. 2 Mass store I/O passes N related to binary exponents IBEX, ICEX and M, for rou- 
tine CMFFT. Increase diagram values of M and N by one to obtain working M and N for 
routine RMFFT. 


where TCA is the FFT computation overhead per block and TMA is the average mass store access time 
per block. TCO and TMO are the limits for large block size. TCA and TMA set a useful lower limit 
for block size. In the examples above TM is mainly the result of TMA, or system block access time. 
Increasing block size reduces TM but with a probable increase in N (depending on M and ICEX). TC 
is mainly due to TCO, TCA becoming important typically when IBEX < 5. To determine the most 
efficient parameter combination, begin by choosing ICEX as large as possible, use Fig. 2 as a guide in 
choosing IBEX but follow this up by timing comparisons over a range of IBEX. 


The program was written to test the sorting algorithm [1] in practice. At the same time a useful and 
efficient Fourier program has been obtained, operating with a very general set of parameters. Some 
parameter combinations can lead to short cuts, not detected or implemented here (see section 5). 
Efficiency in a particular case can no doubt be improved by some recoding, at the expense of a loss in 
generality or increase in code complexity. 


Hopefully, this program will stimulate designers not only of fast transforms but also of general pur- 
pose systems. Sorting by index bit manipulation should be considered an important concept for new 
machine architecture. Index bit-reversal is simple to achieve in hardware and, in its generalized form 
[1], is a symmetric permutation (allowing in-place operation) which forms the basis of a number of 
useful permutations, not exclusively associated with the FFT. 
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5. Discussion of Algorithm 
5.1 General 


The structure of the mass storage FFT program is shown in Fig. 1. When using real data, routine 
RMFFT calls CMFFT with a half length complex array before unscrambling by MFRCMP. If neces- 
sary, routines CMFFT and below may be overlaid with routines MFRCMP and MFRLOD. CMFFT 
controls the sequence of mass store accessing for in-core computation and sorting of the complex FFT. 
Routines MFCOMP and MFRCMP carry out the FFT computation and unscrambling of data in core 
store. MFREV, controlled by routine MFSORT, carries out a bit-reversed permutation on elements in 
core store or on whole blocks in mass store. MFLOAD and MFRLOD load and unload core store for 
the in-core routines, calling on MFREAD and MFWRIT (section 7). MFINDX defines a virtual per- 
mutation of the mass store array during the complex FFT operation. MFSUM sums elements in the 
dimensioning list MEXA( ), sorting them if necessary. Routine MFPAR is discussed in section 3.4 and 
routine DMPERM in section 3.6. 


The main problem in writing a mass store FFT program is to devise an efficient means of accessing 
widely spaced data elements for use in the FFT computation kernel, and for sorting from bit-reversed 
order [3]. Singleton [8] describes an algorithm for accessing a mass store array two blocks at a time to 
carry out each computing pass of an FFT. He organizes mass store accesses in such a way that each 
I/O and computing pass results in a one place cyclic shift of array element index bits. This allows the 
FFT kernel program to access the same elements in core store on each pass, which not only simplifies 
the program but also brings ‘widely separated’ elements in the original array ‘within reach’ of the in- 
core computation. Bit-reversed sorting is done in a similar manner. For an array of 2**M elements, 
the basic method requires (2M—1) I/O passes. : 


With efficient, random access mass store it is possible to use an indexing algorithm to read widely 
scattered blocks, equivalent to a virtual permutation of the array. After an in-core operation the blocks 
are written back in-place, the blocks remaining physically unpermuted by the operation. In [1] an algo- 
rithm is described which uses this method to compute the FFT. The indexing subroutine MFINDX 
computes the required sequence of block indices using an algorithm which is a generalization of the 
Block Indexing Algorithm of [1], p. 303. This is equivalent to a cyclic shift (a number of places) of a 
set of the block index bits. 


5.2 Complex Data 


Unlike Singleton’s algorithm, an attempt is made to load as many blocks together as possible in core 
store. In this way it is possible to include a number of FFT computing passes while the blocks are in 
core store, reducing the number of I/O passes by a factor ICEX—IBEX (see section 4). Routine 
CMFFT uses a slightly modified version of the algorithm of [1], p. 307 for this purpose, doing M—C 
instead of M—B initial computing passes with virtual permutations, followed by final C pass computa- 
tions with direct access, instead of final B pass computations. I/O efficiency is unchanged, but the algo- 
rithm is neater (sizes being 2**B elements per block, 2**C in core store, 2** M total, all complex). 


Routine MFCOMP computes the FFT in core store. But, unlike other FFT routines, the number of 
computing passes NPAS, and the effective initial pass number IPAS, are not fixed but are passed by the 
calling routine CMFFT. Weighting factors are computed using SIN, COS functions on mass store block 
boundaries but recursively within block boundaries, while array element accessing is ordered to minim- 
ize these computations. Multidimensional transforms are achieved by restarting the weighting factor 
sequence over passes corresponding to the exponent of each dimension, requiring no change to the 
order of access of elements. 


Finally, routine MFSORT is called to sort the array from bit-reversed order. The algorithm is a 
slightly modified version of the sorting algorithm of [1], pp. 305-307, to allow the reversal of a more 
general set of index bits. Its structure is very similar to CMFFT, having a first stage using the virtual 
cyclic shift permutation for block access, with pairs of in-core bit-reversed permutations (MFREV) 
instead of FFT computations. (The pairs allow unsymmetric cyclic bit shift permutations to be done 
in-place, in-core). Only one in-core permutation is required on the first I/O pass. These are followed 
by a final bit-reversed permutation of whole blocks, if necessary. 
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In the special case when IDIR=0, MFSORT is called a number of times to individually reverse bits 
corresponding to each dimension exponent. This allows the complex FFT to be done without dimen- 
sion reversal (but note that dimension reversal is required by RMFFT). The dimension shifting sub- 
routine DMPERM, of section 3.6, also uses this feature. In both these cases, I/O is not fully optimized 
and specially written programs could reduce the number of I/O passes by combining some of the 
separate, in-core permutations. 


Bit-reversed index pairs are generated by a very efficient recursion algorithm (routine MFREV). 
The algorithm maintains a hierarchy of ‘reversed’ integers of increasing number of bits, up to 2 less 
than the number being reversed. Incrementing a reversed integer then requires the alternate simple 
addition of a constant or replacement by the next lower incremented reversed integer in the hierarchy, 
recursively. For example, with a 3 bit reversal, the reversed set is (0,4,2,6,1,5,3,7) where the next 
value is obtained either by adding 4 to the last or by replacing it by one of the values (0,2,1,3) of a 2 
bit reversed set. 


This method of reversed series generation is in itself fast, as recursion depths are small on average. 
But, in addition, only quarter length series are generated (—2 bits) and the full length series is derived 
by scaling by 2 and adding offsets. This is equivalent to reversing an integer a(integer—2 bits)b to 
b(reversed integer—2 bits)a where a and b are the outer bits. There are four possible combinations for 
ab, but only those reversals greater numerically than before reversal are required (to prevent nullifying 
double swaps), leaving in general the three offsets 1...0, 0...0 and 1...1. In particular, only the first 
offset is required if the internal (reversed integer—2 bits) is smaller than or the same as before reversal. 
Thus, only valid swap index pairs are generated, saving the unnecessary reversed integer generation of 
some other methods. 


5.3 Real Data 


For transforms in which the initial or final data is real, a half length complex transform of packed 
real data must be unscrambled (or vice-versa) by routine MFRCMP. The method relies on complex- 
conjugate symmetry in the transform of real data [4]. Calculation of indices of multidimensional sym- 
metry is more difficult than in the one-dimensional case. To do this a recursion algorithm is used, 
which is most easily described by the following FORTRAN program (for two dimensions, N1 by N2 
elements): 


L2=N2/2 + 1 
DO 1 J2=1,L2 
K2=N2+2—J2 
IF(J2.EQ.1)K2=1 
L1I=N1 
IF(J2.EQ.K2)L1=N1/2 + 1 
DO 1 Ji=1,L1 
K1=N1+2-—J1 
IF(J1.EQ.1)K1=1 
C (Now have A(K1,K2) and A(J1,J2) as a pair with conjugate symmetry) 
1 CONTINUE 


Note is taken of the special cases which exist when the J index is 1 (FORTRAN) and when the J and K 
indices are equal. The general case follows by repetition of the code between the two DO statements, 
replacing J2 by Jj etc. In routine MFRCMP arrays JAYA(4) etc. are used for this purpose. To increase 
the number of dimensions allowed in RMFFT (section 3.3), increase the array sizes. To help visualize 
‘the result, the index relationships (FORTRAN index —1) for an 8 by 4 array are as follows (r for real 
IF (index(K1,K2).EQ.index(J1,J2)), where index(J1,J2) = J 1+02- 1)* N1 for example, c for complex 
conjugate): 


r00 01 02 03: = 104-- 203....c02...cOl 
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The array above represents the transformation and transposition of a 4 by 8 real array (imaginary 
part zero). However, we may also consider it to represent the scrambled transform of an array of ini- 
tially packed real data, occupying alternately real and imaginary elements, that is initially 8 by 8 real 
values. In this case the result is only half length, element pairs such as 15 and cl5 above containing 
information which can be unscrambled to give a new element 15 and an element 33 in place of cl5. In 
addition, new elements cl5 and c33 are derived. Thus the array is expanded to an 8 by 8 full transform 
of the real data: 


clf: -cl7:...el6—: 2lS.. -2]45..cld.2¢622..: ch! 


Note that the same indexing algorithm applies in both examples (with different N2) but during 
unscrambling and expansion only the half-length symmetry is used (N2=4), corresponding expanded 
rows in the new array are obtained by a direct offset (4 in this case). Thus, by half-length symmetry, 
initial rows 1 and 3 are accessed together for unscrambling; these are replaced by unscrambled rows 1 
and 3 of the full array while new rows 5 and 7 are also computed and added to the array. It is this abil- 
ity to add data beyond the existing data which makes dimension reversal essential for in-place computa- 
tion in the real mass store FFT routine RMFFT. 


The complete array is ‘fully redundant’, nearly half the elements being complex conjugates of the 
other half. If the last three rows of the example are left out the result is ‘partially redundant’, since 
row 0 and row 4 still have some internal redundancy. To eliminate all redundancy, rows 0 and 4 can be 
merged (c43,c42,c41 replacing c03,c02,c01), and pairs of real elements combined as single complex ele- 
ments (r40 as the imaginary part with r00 real, r44 with r04). 


The exact algorithm by which an array, with IPAK=0 or —1, can be restored to full redundancy is 
given by the half-length symmetry program above, putting: 


A(J1,J2+N2) = CONJG( A(K1,K2)) 
and A(K1,K2+N2) = CONJG( A(J1,J2)), | IF(2.NE.1), 


with special provision IF(J2.EQ.1) when IPAK=—1, below 


A(K1,K2+N2) = A(K1,K2) 
A(J1,J2+N2) = CONJG( A(K1,K2)) 
and A(K1,K2) = CONJG( A(J1,J2)) 


and IF(index(K1,K2).EQ.index(J1,J2)) when IPAK=—1, values are real, below 


A(J1,J2+N2)= CMPLX( AIMAG( A(J1,J2)), 0.) 
and A(J1,J2) = CMPLX( REAL( A(J1,J2)), 0.) 


the general case follows with Jj and Nj instead of J2 and N2 and as before with repetition of the code 
between DO loops. 


6. Test Programs 


A universal test program is provided which sets up, for a given M, exhaustive permutations and 
combinations of the different parameters IBEX, ICEX and NDIM from 1 to 3 dimensions. Mass store 
is simulated, through dummy routines MFREAD and MFWRIT, by a core store array so that the pro- 
gram is independent of system I/O. 3 


The program stores a pseudo random number sequence in the simulated mass store, transforms this 
by the mass store FFT routines and compares the result with a discrete Fourier transform computed 
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naively by routine NAIVE. An inverse mass store FFT is called and the result compared with the ini- 
tial data. Maximum differences are noted and various levels of data and difference print-out are avail- 
able through parameter IPRINT. The test is considered successful if maximum differences are explain- 
able in terms of machine round-off errors (see Table 1). 


Because the tests are exhaustive and because of the naive DFT computation the program is quite 
slow when the overall array size exponent M is other than very small. For example, in a CYBER 76 
computer with M=5 the program takes 0.3 seconds while with M=10 the program takes about 800 
seconds. Less exhaustive testing can be done by replacing program DO-loop variables by fixed vari- 
ables (see program comments). 


Example test programs are also included for accessing mass store in two specific computer systems. 
These are a CYBER 76 (true mass store and LCM ‘static’ store) and a PDP 11. Mass store I/O rou- 
tines for other systems should be easily devised based on these examples and the comments of section 
: a 


7. System-Dependent I/O Routines 


The system-dependent random access transfers between mass store and core store are handled by 
two subroutines called internally by the FFT routines, thus: 


CALL MFREAD (BUFA,NB,JB) Read block into core store 
and CALL MFWRIT (BUFA,NB,JB) Write block from core store 


where BUFA is the core store address for the start of the transfer, NB is the number of real elements 
to be transferred and JB is the desired block or record number. The range of JB is: 


for CMFFT or RMFFT if IPAK=+1 
1 < JB < 2**(M—IBEX+1), 


for RMFFT if IPAK=0 


1 < JB < 2**(M-—IBEX) + 2**(M—MEXA(NDIM)-—IBEX-+1), for 
MEXA(NDIM)+IBEX < M+1, 


or 1 < JB < 2**(M-—IBEX) + 1 for MEXA(NDIM)+IBEX > M+1, 
(the MEXA list in the order after the transform) 


for RMFFT if IPAK=—1 
1 < JB < 2**(M-—IBEX). 


In addition, the I/O subroutines must know what file to access and any starting block offset, which 
may be given through COMMON variables, for example. The routines expect the mass store file to be 
defined and opened. The user program may, of course, call MFREAD and MFWRIT itself to load 
mass store with data or read the transform result. 


For example, in PDP 11 FORTRAN a random access occurs with 


SUBROUTINE MFREAD (BUFA,NB,JB) 

C READ BLOCK, INDEX JB, FROM MASS STORE TO BUFA, NB REAL VALUES > 
REAL BUFA (NB) 
COMMON/FFTCOM/LUN : | 
READ (LUN’JB)(BUFA(I),I=1,NB) or READ (LUN’JB) BUFA 
RETURN 
END 


and similarly for subroutine MFWRIT. 


As discussed in sections 3.4 and 3.5 it may be useful to call MFREAD/MFWRIT with a ‘record’ 
length NB different from 2**IBEX. This can give a user more logical access to a multidimensional 
array, in ‘records’ of first dimension length, for example. Some systems allow a redefinition of the file 
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structure, in which case the problem is solved. Others allow a ‘word-addressable’ file structure in which 
an I/O transfer may start at any word in the file. In this case, MFREAD/MFWRIT can include the fol- 
lowing: : 


INDEX = (JB—1)*NB + 1 | 
(Transfer between file real element indices (INDEX) and (INDEX+NB-—1)) 


The dummy I/O routines of the universal test program of Section 6, and the CYBER Extended 
Core/LCM and PDP 11 Macro routines, include this feature. 


Subroutines MFREAD and MFWRIT allow the user considerable scope for modifying the operation 
of the FFT routines. In the universal test driver program (section 6) the subroutines simply copy data 
from one core store array to another. In some systems external core store can be accessed efficiently in 
blocks, which allows this to be used as a fast, static mass store. Using different files on the first I/O 
pass for MFREAD and MFWRIT allows data to be copied automatically from a source file to a working 
and result file. In this case care must be taken to allow only a single read of each block of the source 
file, any subsequent read, even on the ‘first? I/O pass, being from the working file (important during 
complex-to-real transformation by routine RMFFT). 
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Table 1 
MASS STORE FFT TEST 
IPRINT = 1 (1=COPIOUS, 0=MAX DIFFS,-1=OVERALL) 
IRMF = -1 (0O=CMFFT TEST,-1=RMFFT TEST) 
NDIM = 1 ISGN = -1 IBiRoe>4.> Teese =. I1Ccet =.-4 SPARS 14 
MEXA() = 5 
INDEX FFT NAIVE DIFF 


0 0.1608 02 —2. 031608 (02° 0. 0. 0. 
+ -0.132E 01 -0.6388 00..-0.132E 01.-0.6368 OC. <0. t0e-06->.75E-07 
22330. 9718 Gi 30.4178 GO=s-0 9772-00 .-0.417E 20 0.135 =064 0 .86E-07 
3 -0.146E 00 -0.306E 00 -0.146E 00 -0.306E 00 -0.13E-07° 0.34E-07 
4 -0. 1876 00 =G. 2672 GC "0.7875. 00 -—0.287E. 00 O.37E-07 = 8228-07 
5 0. 141E O1- =8.7 238 91 0.141E 01 -0.123E 01 0.15E-07 -0.15E-07 
6 ~-0.9978. 00. 0, 296E 00 --=0.997EF 00... 0.2968 00 --+~0,228-07 +0-2675-07. 
7 -=~Q. 3198. 00: —-0. 1648 01 -0@. 3198. 60 --0.184E O71 0.18E-06 -0.36E-06 
8 —0. 2678 00 -0. 134 OF -0.2618 00 -0.1345 01 .-0.178-06..-0.30E-07 
9 0.1818 04 0. 7665 01 DO, 16te 01-0. T66E 01... -0. 228-06 -0. 758-07 
10. --0,3 712-02. -0. 1042 01 -0.11178 01. -O.104E 01 . -0.75E-07 <0:°968-06 
11 -0.17 768-07 =O. 4525 00 =0.17768 01. -0.4525 00. -0.308-07 0. 228-06 
12 0.874E 00 -0.210E 01 0.874E 00 -0.210E 01 0. 138-06 --0:..308-07 
13 0.8758 00-0. 2208: 01 0.875E. 00 -0.220E 01 -0.898-07 -0. 18E-06 
14 -0.2555 01 -0.138E.01 ~0,25528-0t -G. 7325 01 0.15E-06 0.42E-06 
15 0.204E 00 - 136E.06 0.204E 00 0.738E 00 0.16E-06 -0.47E-06 
16 -0.844E 00 -0.844E 00 -0.330E-07 0. 6. 33E-07 
17 0.204E 00 -0.738E 00 0.204E 00 -0.738E 00 0.16E-06 0.22E-06 


0 
0. 
0 
18. -0.2556 01 -0.4388 01 =8.2558 01. 0.i2er © = -0 302-07 -0 .63E-06 
+S 0.8758: 00 -0.2208 01 0.875E 00 0.220E 01 -0.27E-06 -0.60E-07 
20 0.874E 00 0.210E 01 0.874E 00 0.210E 01 -0.80E-06 0.60E-06 
21 -0.176E 01 0.452E 00 -0.176E 01 0.452E 00 0.55E-06 0.46E-06 
22 -0.1118 01 0.1048.61 --0.1T1E 01 0.1042 01 0.635-06 0.39E-06 
ao 0. 181E 01. .-0.. 1668 -01 0.181E 01 -0.166E 01 0.60E-07 -0.31E-06 
24. -0.281E 00° 0.1348 01 -0.281E 00 0.134E O01 . -0.33E-06 °§0.30E-07 
25 -0.3192 09 0.7642 01  -0.319E 00 0.1642 01. -0.17E-05 —0. 63E-06 
26 -0.997E 00 -0.296E 00 -0.997E 00 -0.296E 00 -0.89E-07 -0.55E-06 
ar 0.141E 01 0.123E 01 0.141E 01 0.123E 01 0.30E-06 0.28E-06 
28 =-0.187E GO 0.2878 00 -0.187E 00 0.287E 00 0.1718-05 8. 125-05 
29 -0.146E 00 0.306E 00 -0.146E 00 0.306E 00 -0.75E-06 -0.21E-06 
30° -0..971E 00° 0 ATTE 00-0. 9718 00. 0.47 7E 00°. -0.290E-65 -0.53E-06 
31  =0. 1328 01. 0.636200 .-0. 1328-01. O.G638E 00 - --0.72E-06 0.3 72-07 
MAX DIFF 0.1989E-05 
NDIM = 1 ISGN = -1 IDIR = -1 IBEX =. 2 ICEX = 4 IPAK = 1 
MEXA() = 5 
MASS STORE FFT TEST 
IPRINT = 1 (1=COPIOUS,0=MAX DIFFS, -1=OVERALL) 
IRMF = -1 (0O=CMFFT TEST,-1=RMFFT TEST) 
NDIM = —t— 25GN =--1 IDIR: = 1. TBER-= -2-.-FCEx = 4 TRPAR = 1 
MEXA() = 5 
INDEX FFT/2**M INPUT DIFF 
0 0.194E 00 0O.194E 00 0.373E-08 
1 0.958E 00 0.958E 00 0.745E-08 
P 0.135E 00 0.135E 00 0.745E-08 
3 0.696E 00 0.696E 00 0.149E-07 
4 0.233E 00 .6.233E. 00 0.186E-08 
5 0.465E 00 0.465E 00 0.745E-08 
6 0.7378 00. 8.737500 o. 
7 0.516E 00 0O.516E 00 0.149E-07 
8 0.897E 0G 0.897E 00 0.745E-08 
9 0.981E-01 0. 981E-01 0.373E-08 
10 0.963E. 00 0.9635 00 0.149E-07 
11 0.461E 00 0.461E 00 0. 
12 0.186E 00 0.186E 00 0.745E-08 
+3 0.856E 00 0.856E 00 0.745E-08 
14 0.815E 00 0.815E 00 0.745E-08 
72 0. S325 00 — 0.5328 00 0.149E-07 
16 0.100E 00 0.100E O00 0.373E-08 
17 0, 739E 00 0.7398. 00 0.745E-08 
18 0.292E 060 0.292E 900 0. 
19 0. /2/6 00° 0.7272 00 0.149E-07 
20 0.639E.00 0.639E 00 as 
21 0.747E 00 0.747E 00 0.745E-08 
Ba 0.3935 00 -0.3932 00 0. 
23 0.474E-01 0.474E-01 0.373E-08 
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(Continued) 
24 0.803E 00 0.803E 00 0.745E-08 
25 0.879E 00 0O.879E 00 0 
26 0.120E 60 ©..4208 -00 0. 
27 0.493E 00 0.493E 00 0.745E-08 
28 0.592E 00 0.592E 00 £0.745E-08 
29 0.137E 00 0.137E 00 0.745E-08 
30 0.471E 00 0.471E 00 0.745E-08 
31. 0.630E-01 0.630E-01 0.745E-08 
MAX DIFF 0.1490E-07 
NDIM = 1 ISGN = 1 IDIR = 1 IBEX = 2 ICEX = 4 IPAK= 1 
MEXA() = 5 
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Chirp z-Transform Algorithm Program 


L. R. Rabiner 


Acoustics Research Dept. 
Bell Laboratories 
Murray Hill, NJ 07974 


1. Purpose 


This program can be used to numerically evaluate the ztransform of a sequence along a circular or 
spiral contour beginning at any arbitrary point in the z-plane. 


2. Method 


The program is an implementation of the chirp ztransform algorithm of Rabiner, Schafer, and Rader 
[1]. If we call the input sequence x(n), defined for 0 < n < N —1, then the ztransform of x(n), 
evaluated along the contour z = 4W* (with parameter a) at the set of points 


= AW k= 0A 1 (1) 


X(z,) = X,= FH x(n) A-"w (2) 
n=0 


This program provides a computationally efficient algorithm for evaluating Eq. (2) at the set of M- 
points given by Eq. (1) by implementing it as a discrete convolution [1]. 


The general contour of Eq. (1) is that of a spiral in the zplane. This is more clearly seen if we let the 
complex constants A and W be expressed in the form 


A = Age (3) 
W = Wye” (4) 


As shown in Figure 1, the general z-plane contour begins with the point z = A and, depending on the 
value of W, spirals in or out with respect to the origin. If W)= 1, the contour is an arc of a circle. 
The angular spacing of the samples is 27r¢p. The equivalent s-plane contour with s = (In z)/T begins 
with the point 


5) = 79 + jag= In A (5) 


and the general point on the s-plane contour is 
eae k(Ao+jAw) = = On 4fawe ofk% <1 (6) 


Since A and Ware arbitrary complex numbers we see that the points s, lie on an arbitrary straight line 
segment of arbitrary length and sampling density. 

3. Program Description 

3.1 Usage 


The program consists of the CZT subroutine, two associated subroutines (RECUR and DECUR), and 
an FFT subroutine (FFT842). The user passes the data sequence and all the parameters required to 
specify the contour and the appropriate transform sizes for processing, and the subroutine returns the 
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(a) 


Fig. 1 An illustration of the independentent parameters of the CZT algorithm. (a) How the z-transform is evaluated on a 
spiral contour starting at the point z = A. (b) The corresponding straight line contour and independent parameters in 
the S-plane. 


z-transform of the sequence in the same array in which the input sequence was passed. The FFT sub- 
routine is a radix 2 complex input FFT [2]. As such the CZT routine requires that all relevant FFT’s 
are for sequences whose length is a power of 2. 


3.2 Description of Parameters of CZT Subroutine 


The basic call to the CZT subroutine is of the form 
CALL CZT (XR, XI, NDATA, NOPTS, DLTSIG, DLTOMG, WTR, WTI, SIGO, OME, NTR, NFFT, FS) 
where 


XR Array of size NFFT. On input XR contains the real part of the input data (Re[x(n)]); on 
output XR contains the real part of the transform (Re[X,]). 


xi Array of size NFFI. On input XI contains the imaginary part of the input data 
(Im[x(7)]); on output XI contains the imaginary part of the transform (ImLX;,]). 


NDATA _ The number of points in the input sequence (JN). 

NOPTS The number of points in the output sequence (M). 

DLTSIG Increment in sigma along the CZT contour in Hz. (DLTSIG = Ag/27z in Figure 1). 
DLTOMG Increment in omega along the CZT contour in Hz. (DLTOMG = Aw/27 in Figure 1). 
WTR Array of size NFFT which holds the real part of the transform of wer), 
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WTI Array of size NFFT which holds the imaginary part of the transform of W~"/2), 

SIGO Initial value of o of CZT contour in Hz. (SIGO = o)/27z in Figure 1). 

OME Initial value of wm of CZT contour in Hz. (OME = @,)/2z7 in Figure 1). 

NTR Rotation factor on input data to make input appear to begin at n = —NTR instead of n = 
0. (Set NTR = 0 to avoid using this option.) 

NFFT Size of FFT performed internally in CZT subroutine. NFFT is a power of 2 that is not less 
than (NDATA+NOPTS-1). 

FS Sampling frequency of data in Hz. 

FFT842 FFT subroutine with calling sequence CALL FFT842(IT,NFFT,XR,XI) where 

IT = 0 for direct FFT 


= | for inverse FFT 


NFFT = Size of FFT 
XR = Array of length NFFT - real part of data 
XI = Array of length NFFT - imaginary part of data 


3.3 Dimension Requirements 


The DIMENSION statement in the calling program should be modified according to the requirements 
of each particular problem. The required dimensions of the arrays are 


XR must be a power of 2 no less than NDATA + NOPTS —1 
XI same as XR 
WTR same as XR 
WTI = same as XR 


3.4 Summary of User Requirements 


(1) Specify input sequences XR and XI, and parameters NDATA, NOPTS, DLTSIG, DLTOMG, 
SIGO, OME, NTR, NFFT, FS and supply FFT routine FFT842. 


(2) Call CZT. 
(3) Output is obtained in arrays XR and XI. 


4. Test Problem 


Compute the z-transform of a delayed impulse along any CZT contour. The test program (See appen- 
dix) accepts as input from the teletype the CZT parameters NDATA, NOPTS, FS, SIGO, OMEO, 
DLTSIG, and DLTOMG, as well as the delay parameter IDEL to form the sequence 


x(n) = 1 n = IDEL 
=( otherwise 


The test program prints the input information, determines the FFT size, and prints the real and ima- 
ginary parts of the transform of the sequence. 


The subroutines CZT, RECUR and DECUR are given in the appendix. Tables 1-4 give outputs of the 
test program for 4 test runs. Figures 2-5 show plots of the output sequences (both real and imaginary 
parts) for these test runs. The parameters used in these runs are printed in Tables 1-4. 
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0 FREQUENCY (Hz) ae 


Fig. 2 Output sequences for the input signal x(n) =1, n=1 and x(n) =0 for all other n, evaluated on the contour 


27 


D iar 
Z, = € 100 $ k= 0, | See S 


X (z)= 27! 
0.809 


Re k(z, 1] 


FREQUENCY (Hz) inte 


-0.949 


1000 I990 
FREQUENCY (Hz) 


Ss eee ce 
J J 
Fig. 3 Output sequences for the same input as Fig. 2, evaluated on the contour z, = e !%e 19 | k =0,1....,99. 
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X (z)2#z~4 
1.0 
0 Re [x (z,)] 
es 9900 
FREQUENCY (Hz) 
i 
0 Im[x(Z,)] 
-{ 
0 9900 


FREQUENCY (HZ) 


Fig. 4 Output sequences for the input signal x(7) =1, n = 4, and x(n) =0 for all other n, evaluated on the contour 
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ek 
z= e 10 4 =0,1,...,99. 
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Fig. 5 Output sequences for the same input as Fig. 4, evaluated on the contour z, = e 10 ¢@ 1000 @~k/10000, « — Q,1,...,99. 
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Table 1 


NO OF DATA POINTS= 100 
NO OF OUTPUT POINTS= 100 
DELAYED IMPULSE AT SAMPLE NO= 1 
SAMPLING FREQUENCY= 10000.000 
INITIAL VALUE OF SIGMA= 
INITIAL VALUE OF OMEGA= 


-000 
.000 


INCREMENT IN SIGMA ALONG CZT PATH= -000 

INCREMENT IN OMEGA ALONG CZT PATH= 100.000 

REAL PART OF OUTPUT SEQUENCE 
.99996E O00 -99804E 00 ~-99207E 00 -98224E 00 -96854E 00 
-95102E 00 -92971E 00 -90479E 00 -87627E 00 -84426E 00 
»80895E 00 .77048E 00 .72889E 00 -68450E 00 .63739E 00 
S87 7728 00 JS979E 00 -48173E 00 -42575E 00 -36811E 00 
.30899E 00 -24869E 00 -18736E 00 -125298,..90 .62772E-01 
-50576E-05 - .6279TE +0". -~£P2oS1E 00 -18733E 00 -24863E 00 
-30901E 00 -~J36810ER 00 .-.@2576E 00 -48173E 00 2 3980E .00 
-58/7792 00 - .63741E 00 - .68454E 00 -72891E 00 -77045E 00 
-80895E 00 -.84428E 00 -.87622E 00 -90478E 00 -92974E 00 
-95097E 00 =. 96852E 00 ~ geen O00) -I99207E 90 -99798E 00 
-99994E 00 -.99796E 00 = YSLZOSE “OU -98220E 00 -96853E 00 
-95101E 00 - -92968E: 00 -.90479E 00 -87625E 00 -84426E 00 
.80894E 00 ~ I TOS0E «00 -.72888E 00 -68448E 00 -63740E 00 
5677 7E-00 -.53580E 00 - .48173E 00 -42573E 00 -36810E 00 
- 30900E 00 -.24868E 00 -.18736E 00 ~12532E 00 -62778E-01 
-17870E-04 -62786E-01 »12533E 00 -18734E 00 -24866E 00 
-30901E 00 - 36811E 00 -42574E 00 -48172E 00 »93580E 00 
-98775E 00 -63739E 00 -68450E 00 ~/2891E 00 -77047E 00 
.80895E 00 -84426E 00 .87620E 00 -90476E 00 e929 73E 00 
-95097E 00 -96854E 00 ~98226E 00 -I92Z07E 00 .99799E 00 

IMAGINARY PART OF OUTPUT SEQUENCE 
-12159E-04 - .62797E-01 +. teooae 00 -18734E 00 -24865E 00 
-30900E 00 =, 300 10H 00 -.42575E 00 -48173E 00 -53582E 00 
9677728. 00 -.63742E 00 -.68451E 00 -72894E 00 .77043E 00 
-80894E 00 - .84426E 00 - .87624E 00 -90476E 00 J2975en OG 
-95097E 00 -.96853E 00 -.98224E 00 -99207E 00 <99795E 00 
-99997E 00 - .99796E .00 a meee, OD -98218E 00 -96854E 00 
-95099E 00 +, 929672 00 -.90477E 00 -87628E 00 -84427E 00 
-80898E 00 -.77048E 00 -.72892E 00 -68449E 00 -63740E 00 
-58776E 00 -.53582E 00 -.48172E 00 -42576E 00 - 36808E 00 
-30900E 00 -.24868E 00 -.18735E 00 129308 ..00 -62774E-01 
-597220E-05 -62794E-01 ~T2Z93TE 00 -18735E 00 -24864E 00 
-30902E 00 -36810E 00 -42573E 00 -48173E 00 -53580E 00 
.58774E 00 -63740E 00 -68450E 00 -I2891E 00 -77046E 00 
-80899E 00 -84426E 00 -87627E 00 -90477E 00 -92974E 00 
-95098E 00 -96855E 00 -98224E 00 -99208E 00 1 99796E° 00 
-99994E 00 -I99/97E 00 -99208E 00 -98218E 00 -96853E 00 
-95100E 00 . 929725 09 .90477E 00 -87627E 00 -84424E 00 
-80895E 00 -77049E 00 -72891E 00 -68449E 00 -63741E 00 
-98775E 00 -53578E 00 -48171E 00 ~-42573E 00 -36807E 00 

-24867E 00 -18735E 00 -12530E 00 -62779E~-01 


-30898E 00 
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Table 2 


NO OF DATA POINTS= 100 

NO OF OUTPUT POINTS= 100 

DELAYED IMPULSE AT SAMPLE NO= 1 
SAMPLING FREQUENCY= 10000.000 
INITIAL VALUE OF SIGMA= .000 
INITIAL VALUE OF OMEGA= 1000.000 
INCREMENT IN SIGMA ALONG CZT PATH= 
INCREMENT IN OMEGA ALONG CZT PATH= 
REAL PART OF OUTPUT SEQUENCE 


.80154E 


.80899E 00 .80530E 00 

.79014E 00 .78624E 00 - /8237E 
.77048E 00 .76648E 00 . 16239E 
-75010E 00 .74593E 00 .74176E 
-72893E 00 .72463E 00 . 72026E 
-70711E 00 .70261E 00 -69814E 
.68448E 00 .67993E 00 .675298 
-66129E 00 -65654E 00 -65181E 
.63737E 00 .63254E 00 ~6276968 
.61286E 00 -60789E 00 -60291E 
.58776E 00 .58266E 00 caliaes 
-56207E 00 -55684E 00 -S5T61E 
.53576E 00 -53046E 00 .52514E 
.50903E 00 -50360E 00 .49817E 
-48172E 00 .47622E 00 -47066E 
-45396E 00 -44835E 00 -44272E 
.42578E 00 -42007E 00 .41433E 
.39713E 00 .39136E 00 (so S0EE 
.36809E 00 .36224E 00 - 329637E 
.33874E 00 .33278E 00 .32685E 

IMAGINARY PART OF OUTPUT SEQUENCE 

.98776E 00 .59286E 00 -59787E 
.61290E 00 .61782E 00 ~62275E 
.63738E 00 -64222E 00 .64701E 
.66128E 00 -66597E 00 -67068E 
.68452E 00 .68910E 00 -69361E 
.70710E 00 -7TISIE 00 s7 19925 
.72890E 00 sFosa20n, 00 .73749E 
.75009E 00 .75421E 00 .75834E 
.77046E 00 .77448E 00 .77840E 
~-79011E 00 -79392E 00 .79774E 
.80899E 00 .81267E 00 -81629E 
.82702E 00 .83054E 00 .83405E 
.84425E 00 .84761E 00 -85090E 
.86072E 00 .86388E 00 .86703E 
.87624E 00 .87925E 00 .88223E 
.89094E 00 .89376E 00 -89657E 
.90477E 00 .90746E 00 -91007E 
.91769E 00 .92015E 00 -92261E 
.92970E 00 .93198E 00 .93422E 
.94078E 00 .94292E 00 .94501E 
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000 
10.000 


, 797795 
.77843E 
- 158335 
.73750E 
<715932 
-69359E 
.67067E 
-64701E 
-62275E 
7997895 
-57240E 
-54633E 
-o 1981S 
-49270E 
-46512E 
-43709E 
-40861E 
<37 9768 
, 390515 
-s2091E 


-60291E 
.62764E 
-65181E 
.67531E 
-69812E 
.72027E 
.74175E 
- 76238E 
. 78235E 
.80152E 
-81991E 
.83747E 
.85420E 
.87012E 
.88519E 
.89932E 
.91264E 
.92500E 
.93644E 
-94702E 


. /9396E 
-77450E 
-75420E 
.73326E 
<7 11520 
-68907E 
-66596E 
-64222E 
-61780E 
-59285E 
~96723E 
.54106E 
.51442E 
-48723E 
-45954E 
-43145E 
.40287E 
-2/73935 
-34463E 
.31496E 


.60790E 
~-63255E 
-65654E 
-67995E 
. 70260E 
.72462E 
~74591E 
. 76645E 
- 78623E 
-80529E 
-82345E 
-84089E 
-85748E 
.87321E 
.88807E 
.90206E 
-91516E 
-92739E 
-93862E 
-94901E 


Za 
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Table 3 


NO OF DATA POINTS= 100 
NO OF OUTPUT POINTS= 100 
DELAYED IMPULSE AT SAMPLE NO= 4 
SAMPLING FREQUENCY= 10000.000 
INITIAL VALUE OF SIGMA= 
INITIAL VALUE OF OMEGA= 


-000 
.000 


INCREMENT IN SIGMA ALONG CZT PATH= -000 
INCREMENT IN OMEGA ALONG CZT PATH= 100.000 
REAL PART OF OUTPUT SEQUENCE 
-99998E 00 -96855E 00 -87630E 00 
-30901E 00 -62805E-01 =2#1873756.090 
-.80899E 00 -~«929755.,09 -.99206E 00 
-.80894E 00 -.63739E 00 - .42578E 00 
- 30900E 00 -53579E 00 .72896E 00 
.99997E-09 -96853E 00 -87624E 00 
- 30900E 00 -62806E-01 -.18736E 00 
-.80897E 00 - .92976E 00 - .99206E 00 
-.80899E 00 -.63739E 00 -.42578E 00 
-30899E 00 -53581E 00 »7/2895E 00 
-99998E 00 -96853E 00 .87627E 00 
-30900E 00 -62789E-01 -%18735E.00 
-.80896E 00 -.92974E 00 -.99204E 00 
-.80898E 00 -.63740E 00 -.42577E 00 
-30902E 00 -53581E 00 -72895E 00 
-99995E 00 -96855E 00 -87626E 00 
-30899E 00 -62789E-01 -.18738E 00 
-.80896E 00 ~«92975E 00 -.,99203E.00 
-.80897E 00 -,637378,.090 -.42579E 00 
-30898E 00 -53582E 00 -72895E 00 
IMAGINARY PART OF OUTPUT SEQUENCE 
-.17881E-05 -.24868E 00 -.48173E 00 
-.95103E .00 ~.99799E 00 ~s98225E .09 
-.58778E 00 -.36811E 00 -.12530E 00 
-58775E 00 -77050E 00 -90477E 00 
~-95101E 00 .84428E 00 -68453E 00 
.54791E-06 -.24870E 00 - .48172E 00 
-.95100E 00 -.99792E 00 -.98227E 00 
-.58776E 00 - .36813E 00 =<12532E -00 
-58778E 00 .-77047E 00 -90478E 00 
-95099E 00 -84430E 00 -68450E 00 
-.80466E-05 -.24869E 00 -.48173E 00 
-.95100E 00 -.99797E 00 -.98225E 00 
-.58778E 00 -.36810E 00 =212531E .00 
-58776E 00 .77049E 00 -90478E 00 
-95103E 00 -84427E 00 -68452E 00 
- .63641E-05 -.24868E 00 -.48172E 00 
-.95104E 00 -.99796E 00 -.98227E 00 
-.58776E 00 - .36813E 00 -.12530E 00 
-58775E 00 -77045E 00 -90479E 00 
-84429E 00 -68452E 00 


-95096E 00 
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.72893E 
.42578E 
.99205E 
.18739E 
.87624E 
.72895E 
.42577E 
.99209E 
.18738E 
-87627E 
.72894E 
.42580E 
.99203E 
.18737E 
.87625E 
.72895E 
.42577E 
.99205E 
.18737E 
.87628E 


-68453E 
.90481E 
-12533E 
-98225E 
-48172E 
-68453E 
-90479E 


-12534E 


.98223E 
.48173E 
.68450E 
.90480E 
.12532E 
.98228E 
.48172E 
.68449E 
.90478E 
.12532E 
.98222E 
.48174E 
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.53583E 00 
.63738E 00 
.92975E 00 
.62804E-01 
.96854E 00 
.53580E 00 
.63741E 00 
.92975E 00 
.62791E-01 
.96853E 00 
.53579E 00 
.63737E 00 
.92977E 00 
.62795E-01 
.96856E 00 
.53580E 00 
.63740E 00 
.92975E 00 
.62807E-01 
.96854E 00 


.84434E 00 
.77047E 00 
.36811E 00 
.99797E 00 
.24869E 00 
.84428E 00 
.77048E 00 
.36811E 00 
.99800E 00 
.24868E 00 
.84428E 00 
.77043E 00 
.36811E 00 
.99798E 00 
.24868E 00 
.84428E 00 
.77047E 00 
.36809E 00 
.99796E 00 
.24867E 00 
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Table 4 

NO OF DATA POINTS= 100 

NO OF OUTPUT POINTS= 100 

DELAYED IMPULSE AT SAMPLE NO= 4 

SAMPLING FREQUENCY= 10000.000 

INITIAL VALUE OF SIGMA= -000 

INITIAL VALUE OF OMEGA= 1000.000 

INCREMENT IN SIGMA ALONG CZT PATH= 1.000 

INCREMENT IN OMEGA ALONG CZT PATH= 10.000 

REAL PART OF OUTPUT SEQUENCE 
-80902E 00 -82454E 00 -83963E 00 
-88179E 00 -89484E 00 -90730E 00 
-94151E 00 -95174E 00 -96147E 00 
-98695E 00 -99431E 00 -10010E 01 
-10174E 01 -10215E 01 -10251E 01 
» F03T9E 04 -10329E 01 -10331E 01 
~-10302E 01 -10278E 01 -10250E 01 
-10121E 01 -10065E 01 -10003E 01 
-97768E 00 -96883E 00 -95939E 00 
-92722E 00 -91528E 00 -90270E 00 
-86146E 00 -84651E 00 -83102E 00 
-78107E 00 -76338E 00 -74511E 00 
-68732E 00 -66703E 00 -64630E 00 
-98141E 00 -55891E 00 -93599E 00 
-46491E 00 -44048E 00 -41573E 00 
-33953E 00 -31356E 00 -28731E 00 
.20720E 00 -18009E 00 -15279E 00 
-69821E-01 -41955E-01 - | 39895-0901 
. 70276E-01 .98476E-01 .12674E 00 
-2111 28-209 -23918E 00 -26715E 00 

IMAGINARY PART OF OUTPUT SEQUENCE 
-58779E 00 -I6797E 00 -54777E 00 
-48477E 00 -46306E 00 -44096E 00 
.37278E 00 -34942E 00 .32581E 00 
-25341E 00 -22882E 00 -20401E 00 
- t2852E 00 .10304E 00 .77441E-01 
-64782E-05 -25956E-01 -91970E-01 
-13014E 00 -15618E 00 -18220E 00 
-29987E 00 -28559E 00 ~31117E 00 
.38710E 00 -41204E 00 .43682E 00 
-50974E 00 ~wagetes O00 -55708E 00 
-62587E 00 .64813E 00 -67007E 00 
«lasatk O80 -75385E 00 -77374E 00 
-83084E 00 .84888E 00 .86647E 00 
-91614E 00 -93165E 00 -94657E 00 
-98799E 00 .10006E 01 .10126E 01 
-10450E 01 -10545E 01 -10634E 01 
-10861E 01 109235 01 -10979E 01 
-11105E 01 Aaa A -11153E 01 
ee -11168E 01 .11154E 01 
.11067E 01 -11025E 01 -10975E 01 
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-85418E 00 
-91930E 00 
-97053E 00 
-10071E 01 
-10280E 01 
-10328E 01 
102132. 
-99339E 00 
-94927E 00 
-88957E 00 
-81492E 00 
-72637E 00 
-62509E 00 
-51267E 00 
~-39062E 00 
-26084E 00 
21253 1m. 00 
-14026E-01 
.15491E 00 
-29500E 00 


.52714E 00 
-41856E 00 
-30191E 00 
-17903E 00 
-51717E-01 
-78026E-01 
.20816E 00 
-33663E 00 
.46136E 00 
-98035E 00 
-69160E 00 
s19323E 00 
.88351E 00 
-96098E 00 
.10240E 01 
-10716E 01 
-11028E 01 
-11168E 01 
«TIS 3ZE 205 
-10919E 017 


.86828E 00 
-93065E 00 
-97907E 00 
-10125E 01 
-10303E 01 
-10318E 01 
~TQ0170E 07 
-98582E 00 
-93859E 00 
-87579E 00 
.79828E 00 
-70707E 00 
-60350E 00 
-48897E 00 
-36525E 00 
-23413E 00 
.97641E-01 
-42120E-01 
-18306E 00 
.32278E 00 


-50616E 00 
-39581E 00 
<24779E 00 
-15386E 00 
.25894E-01 
-10409E 00 
.23405E 00 
.36194E 00 
-48570E 00 
-60325E 00 
~ 112768: 00 
-81225E 00 
-90015E 00 
-97477E 00 
-10348E 01 
—+O792E 01 
-11070E 01 
oT) Gees ol 
-11103E 01 
-10855E 01 
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Complex General-N Winograd Fourier 
Transform Algorithm (WFTA)* 


J. H. McClellan 
H. Nawab 


Department of EECS 
Mass. Inst. of Technology 
Cambridge, MA 02139 


1. Purpose 


This program computes the discrete Fourier transform (DFT) of sequences whose length JN is a pro- 
duct of relatively prime factors taken from the set {2,3,4,5,7,8,9,16}. The method used in this program 
for computing the DFT is a general-N implementation of Winograd’s algorithm [1], referred to as the 
WFTA [2]. 


2. Method 


In order to set up a common notational framework we first summarize the basic form of the WFTA. 
Consider the computation of the N-point DFT of the sequence x[n], n = 0,1,...,.N — 1. Suppose the 


transform length factors into the product of ~ mutually prime integers, N = N,; x N,x--- XN, 
The WFTA consists of five steps as illustrated in Fig. 1 for the case N = 15 
0 La a nrg Bn seams: F. 0 
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3-POINT Ese 3-POINT 
PRE-WEAVE POST-.WEAVE 


MODULE 5-POINT 5-POINT MODULE 
PRE-WEAVE MULTIPLICATION POST-WEAVE 
MODULE PHASE MODULE 


(6 X 3 = 18 mults) 


Fig. 1 Fifteen-point DFT algorithm decomposed using Good’s mapping to obtain a two- 
dimensional (3x5)-point DFT and Winograd’s algorithms for 3-point and 5-point DFTs. 
The algorithm requires 18=6x3 multiplications which are "nested" between the pre-weave 


and post-weave modules. 


Step 1 The one-dimensional sequence x[n] is mapped into a w-dimensional array s[nj,n ion via 
the Sino correspondence [3,4] 
N 
=< — 
Elk 


* An earlier version of this program can be found in Ref. [4]. 


As N- (1) 
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where <->, denotes residue reduction modulo N. 


Step 2 The pre-weave [6] modules are implemented, one for each factor N,. Each pre-weave 
module contains only additions and subtractions. The size of the data array will expand, in general, 
when passing through the pre-weave elements. | 


Step 3 The data array is multiplied (point-by-point) by an array of real constantst derived from the 
multipliers of the small-N DFT algorithms for N,, i = 1,2,...,u. These constants depend solely on the 
complex exponentials of the DFT, Wy, and are the only multiplications required in the algorithm. 


Step 4 The post-weave [6] modules are implemented for each of the ~ dimensions. Each of these 
modules contains only additions, subtractions and multiplications by j. A different module is required 
for each dimension. After passing cme Re post-weave section, the data array S[k,,k>,...,k a 
shrinks back to its original size, N L x N,xX-+::xX N,. At this point in the computation we have 
obtained the 4-dimensional N, x N> x --- x N, DFT of the array s[nj,n»...,n,]. 


Step 5 A mapping of the uw-dimensional array S[k1,k...,k ul is performed in accordance with the 
Chinese Remainder theorem to obtain the correct one- dimensional DFT, X[k]. This correspondence is 
given by [3,4] 


k= Sh tos tbe E> y (2) 
where the integers s;, satisfy 
$; = 9; mod Ni FEF =F, 2, (3) 
2.1 Initialization | 


In order to have an efficient WFTA program several computations need be performed only once 
when DFTs of the same length are to be calculated over and over. First of all, the factors of N must be 
determined. Since the WFTA is composed of small-N DFT algorithms that have only been developed 
for certain lengths, a check against the allowable factors of N is made. Next, the constant multiplier 
array is formed by multiplying together the pre-stored constants for the individual small-N factors. 
Finally, permutation vectors for the input and output mappings are prepared based on the factors of N. 
This speeds up the actual mapping process because the congruences (3) of the Chinese Remainder 
theorem are solved during initialization. The expense, of course, is the storage to hold the mapping 
vectors. 


3. Program Description 


The WFTA program consists of four subroutines, each of which will be described in the following. 
The flow of control in the algorithm is given in Fig. 2. Documentation of the individual computational 
modules (pre-weave and post-weave) is presented using the annotated flowgraphs of Figs. 3 through 10. 


The WFTA is invoked by calling the subroutine WFTA with the arguments 
N - transform length 
XR(N) - real part of array to be transformed 


XI(N) - imaginary part of array to be transformed. The real and imaginary parts of the transform are 
returned in XR and XI. 


INVRS - a flag to invoke the inverse transform. INVRS = 1 gives the inverse; INVRS = 1 gives the 
forward transform. A factor of N~! is included in the inverse. The inverse is actually 
obtained by modifying the permutation described in step 5 above to map to <—k> instead 
of k. 


INIT - a flag to specify whether the call to WFTA requires initialization. INIT=0 calls for initializa- 
tion, INIT#0 skips the initialization phase. When many DFTs of the same length are to be 
performed, initialization is needed only for the first DFT but not on succeeding calls. See 
subroutine INISHL for details. 


+ Note that the constants are not allowed to be imaginary as proposed by others; rather the post-weave modules contain 
"multiplications" by / to effectively obtain imaginary constants. Thus the need for a flag array is eliminated. 
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IERR -_ is an integer variable that contains, upon completion of a DFT, an error code. If the DFT 
was done successfully IERR=0; if an error occured IERR=—1 or —2. There are two causes of 
the error: (1) the transform length is illegal (i.e., it contains factors outside the set 
{2,3,4,5,7,8,9,16}), (IERR=—1), or (2) the program has not been initialized for the particular 
value of the transform length specified, (IERR=—2). 


The subroutine WFTA calls in turn the subroutines INISHL (if needed), WEAVE1, and WEAVE2. 
The permutations and multiplications are performed within subroutine WFTA. 


NO YES 
SUBROUTINE INISHL 


RETURN 
1ERR =—-1 


DETERMINE 
MULTIPLIER 
COEFFICIENTS 


l 

| 

| 

| 

| 

l 

| 

| 

l 

DETERMINE 
PERMUTATION : 
VECTORS 
| 

ad 


HAS 
N BEEN 
INITIALIZED 
? 


NO RETURN 
IERR =—2 


YES 


SUBROUTINE WFTA 


MAP XR(N), XI(N) 
TO SR( ), SI ) 
VIA INDX1 (N) 


SUBROUTINE WEAVE1 
IMPLEMENT PRE- 
WEAVE MODULES 

SUBROUTINE WFTA. 
ND1*ND2*ND3*ND4 
MULTIPLICATIONS 

SUBROUTINE WEAVE2 


IMPLEMENT POST 
WEAVE MODULES 


SUBROUTINE WFTA 


MAP SR( ), SI( ) 
TO XRIN), X(N) 
VIA INDX2(N) 


Fig. 2 Flow of control in program WFTA. 
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NBASE 
NBASE 1.0 NBASE 
T1 T1 SR2 
NR1 NR1 ~=4.5 7 NR1 
NR2 >—O NR2 ave) >—O NR2 
3-POINT PRE-WEAVE 3-POINT POST-WEAVE 


Fig. 3 Three-point DFT algorithm using Winograd’s decomposition. Labels correspond to 
storage locations in FORTRAN program. 


NBASE O D NBASE 


Fig. 4 Four-point DFT which is also used entirely as a post-weave module in WFTA pro- 
gram. 


NBASE 


NBASE O 


—1.25 


- 1.538841769 


0.5590169944 


0.36327 1264 


0.5877852523 


5-POINT PRE-WEAVE 5-POINT POST-WEAVE 


Fig. 5 Five-point DFT algorithm using Winograd’s decomposition. 


The subroutine INISHL performs those calculations, for a fixed value of N, that need not be done 
on each pass of the algorithm. These include factoring the transform length N into its mutually prime 
factors, calculating the constant coefficients to be used in the multiplication step, and calculating the 
mapping vectors for the pre- and post-permutations of the data to be used in the input and output per- 
mutations. The transform length N is factored into mutually prime factors taken from the set 
{2,3,4,5,7,8,9,16}. Since Ncan have at most four factors from this set, the factors are assigned to the 
integer variables NA, NB, NC and ND as follows: 


NA is assigned the largest value from the set {1,2,4,8,16} that divides N; NB the largest divisor of N 
from the set {1,3,9}; NC the largest from {1,7}; and ND the largest from {1,5}. The product 
NA*NB*NC*ND should equal JN, if the transform length is permissible. In the following, NA will be 
referred to as the first (or outermost) factor, NB the second, NC the third, and ND the fourth (or 
innermost). This reference is with respect to the order of pre-weave modules in the calculations (i-e., 
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NA is the size of the first pre-weave module and last post-weave module).f 


From the factors of N we can now determine via Table 1 the amount of data expansion that will 
occur in the pre-weave phase and, equivalently, the number of multiplications that will be performed in 
the WFTA. For each of the four variables (NA, NB, NC, and ND) the number of multiplications is 
assigned to the variables ND1, ND2, ND3, and ND4, respectively. In order to accommodate the data 
expansion of the WFTA, the working arrays SR(:) and SI(-) must be dimensioned as large as 
ND1*ND2*ND3*ND4. If the present upper limit on the transform size of 5040 is to be reduced, the 
size of the working arrays can be reduced by using Table 1 to determine the data expansion for the 
largest allowable transform size. 


Table | 
Number of multiplications, M(N), needed 
for N-point WFTA 
N MUON) 


ROMINA WHN 
—t ed 
Om PCOKDHWHN 


Note: if NW =1,Xr)*-** Xr, 
then M(N) = M(r,)M(ry)...M(r,) 


The second step of the INISHL subroutine is to calculate the constant coefficients for the multiplica- 
tion routine. These are generated from the coefficients of the individual factors as 


COEF(n,,15,n3,n4) = COA(n,) *COB(n,) *COC(n3) *COD (ng) 


where 
iy > i 2... Maa 
ny = 1,2,..., ND2 
i. = 1, Z,.... ND3 
ng = 1,2,..., ND4 


The array COEF is actually treated as a one-dimensional array in the FORTRAN program, but can also 
be thought of as a multidimensional array in a mixed-radix index representation with the first index 
varying the most rapidly. Thus, the following addresses are equivalent:fT 


COEF(n1,1),n3, nq) _— COEF(n,+ND1 *no>tND1 *ND2 *n3+ND1 *ND2*ND3 *n4) , 
This method of data addressing will also apply to the data arrays SR(-) and SI(:). 


The third step of the INISHL subroutine is the calculation of the mapping vectors INDX1(-) and 
INDX2(-) for the pre- and post-permutations of the data. The pre-permutation of the data is done 
according to the Sino correspondence of Eq. (1). The mapping is implemented using the mapping vec- 
tor INDX1(-) to obtain the highest speed possible. The post-permutation requires the inverse mapping 
of the Chinese Remainder theorem, as in Eqs. (2) and (3). Again, for maximum speed a mapping vec- 
tor INDX2(-) is constructed. 


t This order of pre-weave and post-weave modules will minimize the number of additions in the algorithm in all cases except 
N = 48, 240, 336 and 1680. The increases in additions for these cases is less than 4% compared with the optimal strategy [2]. 
The benefit of the order adopted here is a simpler program structure. 


tt The multidimensional arrays were not used in the FORTRAN program because it was found that a typical compiler produced 
much more efficient code with the one-dimensional arrays. 
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Chapter 1 


7-POINT POST-WEAVE 


Fig. 6 Seven-point DFT using Winograd’s decomposition. Nine multiplications are 


required. 


The three functions of the INISHL subroutine need only be performed once for a given value of 
transform length N. The remaining three subroutines form the computational core of the algorithm. 
The subroutine WFTA maps the data input arrays XR and XI into the working arrays SR and SI using 
the pre-permutation vector INDX1. The arrays SR and SI are treated as one-dimensional arrays of size 
NDI*ND2*ND3*ND4 in subroutine WFTA. Note that the WFTA is NOT computed in-place because 
of the nature of the permutations. | 


NBASEQ 


TO 


8-POINT PRE-WEAVE 


1.0 


8-POINT POST-WEAVE 


Fig. 7 Eight-point DFT algorithm using Winograd’s decomposition. 


Next the subroutine WEAVE1 is invoked to perform the pre-weave modules. The first pre-weave 
module performed is of length NA. If NA=1 the first module is skipped and the next factor, NB, is 
tested. Each factor NB, NC and ND is checked in turn. In WEAVE1 the arrays SR and SI are treated 
as one-dimensional arrays to gain speed in execution. There is no 4-point pre-weave module because 
the 4-point DFT is implemented entirely as a post-weave module. Likewise, the 2-point pre-weave 
module is a 2-point DFT and there is no 2-point post-weave module. Documentation of the code for 
each pre-weave module can be found in the flowgraphs of Figs. 3 - 10. 
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Fig. 9 Sixteen-point pre-weave module. 
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Following the WEAVE1 subroutine the data is multiplied point-by-point by the real array COEF in 
the subroutine WFTA. Then the post-weave modules are implemented in the subroutine WEAVE2. 
The module of length ND (=5 or 1) is done first, then NC, NB and finally NA. The flowgraphs of - 
Figs. 3 - 10 give the post-weave algorithms for the various factors. The multipliers for the 16-point 
DFT algorithm are given in Table 2. Finally, subroutine WFTA performs the post-permutation by 
mapping SR and SI to XR and XI via the mapping vector INDX2(-). This completes the WFTA calcu- 


lation. 


Table 2 
Multipliers for 16-point DFT Algorithm 


1 “coe 
ry ig 


—/2/2 
—0.5411961001 
—1.0 
—1.306562965 
J2/2 
1.306562965 
—0.9238795325 
0.3826834324 
NBASE 
a se 
eg ee = 
T(3) NRA 
= a 
OP IE ag, uote 
T(5) NR2 
ee 
ee = 
ce aR ape a 
ee eS ane 
ie 7 
witness. NR14 
— F(8) ss 
T(9) a7) NR1 


aes wer 
SS aE 


STL a PPO ONR7 


Se Ft) a(6) 


16-POINT POST-WEAVE MODULE 
Fig. 10 Sixteen-point post-weave module. 
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4. Description of Parameters: 


CDA, CDB, CDC, CDD: real arrays that are assigned coefficients of the factors of N during initializa- 
tion (CDA: for outermost factor, CDD: for innermost factor). | 

CO3, CO4, CO8, CO9, CO16: real arrays containing the short-DFT coefficients for factors 3,4,8,9,16 
respectively. 

COEF: real array of size ND1*ND2*ND3*ND4 to which the N-point DFT multiply coefficients are 
assigned during the initial run. 

INDX1, INDX2: integer arrays of size N for storing initial and final permutation mappings respectively. 

INIT: integer variable passed from user program to WFTA. It must be zero on the initial run and 
nonzero otherwise. 

INVRS: integer variable to flag the inverse Gansian (INVRS = 1), otherwise (INVRS = 1) the for- 
ward transform is calculated. 

IERR: integer variable that returns error code indicating successful completion (IERR=0) or abnormal 
termination (IERR=—1 or —2). 

N: transform length 

NA, NB, NC, ND: integer variables for four factors of N (NA: outermost, ND: innermost). 

ND1, ND2, ND3, ND4: integer variables representing the "expansion" of factors NA, NB, NC, ND, 
respectively. See Table 1. 

NMULT: integer variable equal to product of ND1, ND2, ND3, and ND4. 

SR, SI: real arrays of size ND1*ND2*ND3*ND4 to which XR, XI are transferred during initial permu- 
tation. 

XR, XI: real and imaginary parts respectively of complex input data array. The transformed data are 
also placed in these arrays by the final permutation. 


5. Summary of User Requirements 
(1) Specify input sequences XR and XI and parameters N, INVRS, INIT and IERR. 
(2) Call WFTA. For initial call use INIT=0; INVRS = 1 means the inverse DFT will be computed. 


(3) Output is obtained in XR and XI. The error code IERR is zero for successful completion and —1 
or —2 if the value of Nis not permissible or was not initialized. 


(4) After initial call, use INIT~0 for all other calls as long as Nis not changed. 


6. Test Example 


A main program is provided to generate a test waveform whose DFT is known analytically. For this 
purpose a complex exponential signal was chosen 


x[n] = lie! n=0,1,....N—1 (4) 
where JN is the transform length to be tested. The DFT is easily calculated to be 
(4 
ee eee (5) 
| re 


Thus the WFTA can be checked by comparing the results of a DFT computation with Eq. (5). The 
program will print out the maximum absolute and relative differences between the computed DFT and 
the ideal. The test example was run for the case N = 28, forward transform, r = 0.98, and # = 0.5 
with the corresponding input and output given in Tables 3 and 4 respectively. By choosing different 
values for N the various pre-weave and post-weave modules can be exercised. Different inputs can be 
obtained by varying rand ¢. The choice r ~ 1 and ¢ = 0 mod N must be avoided since Eq. (5) does 
not hold in this case. 


7. Summary 


A FORTRAN program for the calculation of the DFT using Winograd’s "nested" algorithm has been 
presented. While every effort has been made to provide a computationally efficient program, some 
compromises have been made to simplify the program structure. For example, the multiplications of 
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the WFTA are all performed in a separate DO loop. A slightly more efficient implementation of the 
algorithm would provide code so that the innermost module of the pre-weave and post-weave computa- 
tions is combined with the multiplications. For example, in Fig. 1 the innermost module is the 5-point 
factor. The advantage of such an approach is that it avoids some memory transfers (loads and stores) 
that are required in our implementation. Furthermore, it reduces slightly the amount of memory 
required for the arrays SR and SI. The disadvantage, of course, is that an additional block of code must 
be provided for each factor in the set {2,3,4,5,7,8,9,16} to allow for the possibility of each factor being 
the innermost. This could conceivably double the amount of program code. 


Another possible improvement in the WFTA that would be quite helpful, would be the develop- 
ment of input and output permutations that could be done "in-place". If this were possible, the arrays 
SR and SI could share memory with XR and XI and only an additional (ND1*ND2*ND3*ND4 — N) 
locations would be needed to accommodate the data expansion of the WFTA. 


If one only requires an algorithm to compute the DFT of real data, a rather simple modification of 
the existing program will exploit the real-valued nature of the input. The input permutation, subrou- 
tine WEAVE] and the multiplications should be modified to remove all calculations involving the array 
SI which will be zero, until subroutine WEAVE2 is called. This will result in a slight improvement in 
the running time of the algorithm for real data. 


As a final comment, it should be pointed out that this program is an initial attempt to produce an 
efficient WFTA program, and it is quite likely that future developments related to Winograd’s algorithm 
will lead to more efficient implementations. 
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1 REAL 
2REAL 
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(9UN)US-( LUN) YS=9L 
(9UN) US+ (LUN) US=LL 
JION+ LUN=8u4Nn 
JAJION+ 9UN= LUN 
JION+SUN=9UN 
JION+ PUN=SUN 
JION+€CUN=P7UNn 
JJION+ ZUN=€CUN 
JION+ LUN=ZUN 
JION+SSVEN= LUN 
JFONSL=LN OLL Od 
GN‘ L=—PN OPL OG 
JION*8=ZdN'IN 
L=aSVaN 

ZAN* LON=440N 


Chapter 1 


HHH HHH HHH HHH HHH HHH HHH HHH HHH HHH HHH HHH HHH HHH HHH HEHEHE HHH HHHEHEHHE HEHEHE HH HEHE 


SINGOW ZAVAM-dudd LNIOd L AHL SLNAWA'IGWI AACOO SNIMOTION AHL 


HHH HH HHH HHH HH HHH HHH HHH HHH HH HHH HHH HHH HHH HHH HHH HEH HHHHHEHEHHHHHHHHHHHHHHKEKE 


00S OL OS (L°UN'ON) AI 
€7TdN IN+HSVEN=ASVEN 
Z7dN IN+ ASVEN=ASVEN 
L+HSVEN=aSVEN 
SL-ZL=(6uUN)IS 
8L-SL=(8uUN)IS 
ZTL-SL=(LYUN)IS 
SL+GL+ZL=(OLUN)IS 
PL-LL=(9U4N)IS 
LL-7L=(SUN)IS 
LL-LL=(7uNn)Is 
LL+PL+LL=(€uN)Is 
(SUN) IS-( PUN) IS=SL 
(SUN) IS+ (PUN) IS=PL 
€L=(LUN)IS 
(SUN) IS-( LUN) IS=8L 
(SUN)IS+ (LUN) IS=LL 
9L=(ZUN)IS 
(ZTUN)IS-(LUN)IS=7L 
(ZUN) IS+ (LUN) IS=LL 


FFT Programs 


OVCUU000 0 


OO0L 
0v6 
0£6 
OL6 


€L+ (ASVAEN) IS=(ASVAEN)IS 
(9UN) IS-(€UN) IS=9L 
(9UN)IS+(€UN)IS=EL 
GL-ZL=(6UN) US 
8L-SL=(8uN) us 
@L-SL=(LYUN) US 
8L+GL+7L=(OLUN) US 
pL-LL=(9UN) US 
LL-pL=(SUN)us 
LL-\lL= (uN) us 
LL+7L+ LL=(€uNn)us 
(SUN) US- (PUN) YS=SL 
(SUN) US+ (PUN) AS=PL 
€L=(LUN) Us 
(8UN) US-( LUN) XS=8L 
(SUN) US+ (LUN) AS=LL 
9L=(ZUN) US 

(ZUN) YS- (LUN) YS=7ZL 
(ZTUN) US+ (LUN) US=LL 
€L+ (ASVAN) US= (ASVAN) US 
(9UN)US-(€UN) US=9L 
(9UN) US+ (€UN) US=EL 
JION+6YUN=0 LUN 
JION+SUN=6uUN 
JION+ LUN=8un 
JION+ 9UN=LUN 
JION+SUN=9UN 
JION+ PUN=SUN 
JION+CUN=PUNn 
JION+ZUN=CUN 
JION+ LUN=ZUN 
JdION+ASVEN= LUN 
LGN‘ L=ZN 016 OG 
ONS L=EN O€6 OG 

GN‘ L=0N O76 OG 
LGN=440N 

L=HSVaNn 
(ON-€GN)*LGN*LL=€7dN'IN 
LON*0L=@dN'IN 


HHH HH HH HH HEH HE HH HH HHH HHH HH HH HHH HHH HHH HHH HHH HHH HEHEHE HEHEHE HEHEHE HH KHEKHEKHHEH KEKE 


2) 

2) 
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@INGOW AAVEAM-Aud LNIOd 6 AHL SLNAWHIMWI ACOO ONIMOTION AHL 2) 
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2) 


HE HE IE HE HE HE HE HE HE HH HHH HH HHH HHH HH HHH HHH HHH HHH HHH HHH HHH HH HEHEHE HEHEHE HEHEHE HEHEHE HEHEHE 


0OL OL OD (6°4HN°AN) AI 006 
€Z7dN IN+ASVEN=ASVEN OVE 
7dN'IN+ ASVEN=SSVEN O€E 
L+aSVEN=aSSVEN OLE 
LL=(LUYN)IS 
(JUN) IS-( LUN) IS=(ZUN)IS 
LL+ (S3SVdEN) IS=(aSVEN)IS 
(TUN) IS+( LYN) IS=LL 
LL=(LUN) us 
(ZUN) US-( LUN) US=(ZUN) US 
L.L+ (€SVEN) US=(ASVEN) US 
(Z7UN) US+ (LUN) YS=LL 
JAJION+ LUN=ZUN 
JION+ASVaEN= LUN 
LGN‘ L=ZN OLE OG 
ONS L=EN O€E OG 
GN‘ L=9N OVE Od 
LGN=440N 
L=aSVaN 
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9L-LL=(LUN) Us 

(8UN) IS-(9UN) IS+(ZUN) IS=EL 
(9UYN)IS-(GUN)IS-(ZUN)IS=SL 
(SUN) IS+ (SUN) IS+(ZUN) IS=9L 
(LUN)US+( PUN) YS+ LL=ALas 
(LUN) YS-(€UN) US+LL=PL 
(VUN)US-(€UN)US-LL=ZL 
(ASVAEN) US+( LUN) YS=LL 
JION+LUN=8uNn 

JJION+9UN= LUN 

JJION+GUN=9UN 

JION+PUN=SUN 

JION+CUN=PUNn 

JION+ZUN=CUN 

JAION+ LUN=ZUN 
JION+HSVEN= LUN 

JAIONSL=LN OLL OG 

- GN‘ L=9N OPL OG 
JION*8=ZdN'IN 

L=aSVaN 

ZON* LON=44ON 


KEK KEKE HEHEHE HEH HEHEHE HH HHH HHH HH HHH HHH HHH HHH HH HHH HHH HHH HHH HH HH HHH HH HHH HH HHH 


HINGOW AAVAM-LSOd LNIOd £ AHL SLNAWH'IdWI AGOD ONIMOTION AHL 


KKH HEHEHE HHH HHH HH HHH HH HHH HHH HHH HHH HHH HHH HHH HHH HH HHH HH HH HHH HH HHH HH HHH HSE 


OO€ OL OD (L°AN*ON) AL 
L+4SVEN=SSVEN 
pUus=(P7UN) US 
TUS=(ZTUN) US 
CTL+EL=(€UN)IS 
CL-€L=(Z7UN)IS 
PL-lLL=(PpuNn)Is 
PL+lLL=(LUYN)IS 
(SUN) US+ (PUN) YS=7ZL 
(SUN) US+ (ZUN)US=PL 
(€UN) IS+ LL= LL 
(€UN)IS-LL=€L 

(LUN) IS+ (ASVEN) IS=LL 
@TL-€L=(€UN) Us 
ZTL+EL=ZuSs 

PL+LL=pus 
PL-LL=(LYN) us 
(SUN) IS+ (PUN) IS=7L 
(SUN) IS+(ZUN) IS=PL 
(€UN)US+ LL=ALL 

(€UN) US-LL=€L 
(LUN)US+ (ASVEN) US=LL 
JAION+ PUN=SUN 
JION+CUN=PuUn 
JJION+ZUN=€CUN 

JAION+ LUN=ZUN 
JJION+HSVaGN= LUN 
JHIONSL=LN OLS Od 
L=aSVaN 

€AGN*ZGN* LGN=d40N 


HH HHH HK HHH HHH HHH HH HH HHH HE HHH HH HHH HHH HH HH HHH HHH HH HHH HHH HHH HHH HHH KH HH HHH HE 


HINGOW ZAVAM-LSOd LNIOd G§ FHL SLNAWAIdWI 3dGOD SNIMOTIO“N AHL 


HEHEHE HHH HH HHH HHH HHH HHH HHH HH HH HHH HHH HH HHH HHH HHH HHH HHH HHH HK HHH HHH HHH HHH HE 


OVVVVN0 NO 


OO0L 
OLS 


OR OR OR OCROROCHS) 


0OOL OL OD (S*4HN°AN) AI 

(9L)L‘°(8)0 NOISNAWIG 

(L)IS‘(L)UYS NOISNAWIG 

VAN ‘€aN‘ZGN‘ LGN‘GN‘ON‘ AN‘ WN NOWWOOD 
(IS‘UYS)7HAVEM ANILNOUGNS 


Sr eee eee ae eee eee Saar er RS Cee esr emp: me ml mee, ah) Ad, cee ee. | et jcpe s i Sae ieeg ma res eap hei tts tc bs teme paem Na N ak e S 


“SHOLOVA ASAHL YOA AACOD AAVAM-LSOd AHL 

SdLNOGXd ANV GN*ON*GN*VN = N HLONAT WHOASNVUL AHL NI 
LNASHadd AMV SHOLOVA HOIHM AS OL SNYOFHO ANILNOU AHL 
“IS GNV YS AYV SAVYYV ONINYOM AHL ‘VLAM AHL JO 
SHTNGOW AAVAM-LSOd AHL SLNAWAIdWI ANILNOUGNS SIHL 


CHAVEM + ANILNOWENS 


aNga 

NYNLaY 
L+4SVEN=aSVEN 
7L=(PUN)IS 
pL=(ZUN)IS 
PL+ZL=(SUN)IS 
(LUN) IS+(ASVEN)IS=(ASVaN)IS 
€L+LL=(LYN)IS 
€L-LL=(€uN)IS 
(7UN) IS-(€UN) IS=ZL 
(7HN)IS+ (€UN) IS=EL 
(VUN) IS+ (LUN) IS= LL 
(PUN)IS-( LUN) IS=$L 
@L= (PUN) US 
PL=(ZUN) US 

pL+ZL= (SUN) us 
(LUN) US+ (ASVEN) US=(ASVAN) US 
€L+LL=(LYN) us 
€L-LL=(€uN)us 
(TUN) US-(€UN) US=7ZL 
(ZUN)US+ (€UN) HS=EL 
(PUN) US+ (LUN) YS=LL 
(PUN) US-( LUN) US=PFL 
JION+PUN=SUN 
JION+€UN=PuUNn 
JAION+ZUN=CUN 
JION+ LUN=ZUN 
JIJION+FSVAN= LUN 
JHONSL=LN OLS OG 
L=aSVaEN 

€AN*Z0N* LGN=440N 


HH EHH HHH HH HH HHH HHH HH HH HH HH HH HHH HHH HHH HHH HHH HHH HHH HHH HHH HEHEHE HHH HHH HEE 


A'INGOW AAVEM-dud LNIOd ¢ AHL SLNAWAIdWI a3aOd ONIMOTIO“N AHL 


KKK HEHEHE HH HHH HHH HHH HH HHH HHH HHH HH HHH HHH HHH HHH HHH HHH HHH HHH HEH KH K HEHEHE HHH 


NYNLAY (G°aN*CN) aI 

7 dN IN+ASVaN=aSVEN 
|+4SVAN=aSVaN 
LL=(LUN)IS 

LL+ (SSVEN)IS=(ASVaN)IS 
CL+PL+LL= Lh 
@L-pL=(LUN)IS 
VL-LL=(PuN)IS 
LL-Z@L=(€uN)IS 
GL-€L=(8uN)IS 


VOUUVOVOUUU0U0UNU YU 


OLS 


O40 0000 


00S 
OvVL 
OLZL 
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L=aSVaGN ON‘ L=EN O€6 OG 


FFT Programs 


(SUN) IS-(LUN)IS-(ZUN)IS=8L 
Q9L=(9UN)US 

(9UN) US+ (PUN) US+LL=LL 
(SUN)US-(PUN)US-LL=ALL 
(9UN) US- (SUN) US+LL=PL 
(OLUN)IS-€L=(€UN)US 
(OLUN) IS+€L=9L 
(€UN)US+ (€UN) US+ (A9SVEN) US=(ASVEN) US 
( LUN) US+ (ASVEN) US=LL 
(€UN)US-(ASVAN) US=EL 
AJION+6UN=0LUN 
JION+SUN=6uUN 

JION+ LUN=8uNn 
JION+9UN=LUN 
JAION+SUN=9UN 
JION+PUN=SUNn 

JION+ €UN=P7uUNn 
JION+ZUN=CUN 

JJION+ LUN=ZUN 
JION+ASVEN=LUN 

LGN‘ L=ZN OL6 Od 


9US=(9UN) US 

GUS=(SUN)US 

TUS=(ZUN) US 

€L-PL=(€uN)IS 
€L+PL=(P7UN)IS 
GL-ZL=(SGUN)IS 
GL+ZL=(ZUN)IS 
9L-LL=(9HN)IS 
Q9L+LL=(LYN)IS 

(8UN)US- (9UN)US+ (TUN) US=EL 
(9UN) US- (SUN) US-(ZUN) US=SL 
(SUN) US+ (GUN) US+ (ZUN) US=9L 
(LUN) IS+ (PUN) IS+ LL=LL 
(LUN) IS-(€UN)IS+LL=FL 
(VUN) IS-(€UN) IS-LL=ZL 
(ASVEN)IS+( LUN) IS=LL 
€L+PL=(€uN) Us 
€L-PL=(P7UN) us 

GL+ZL=Sus 

GL-@L=Zus 

9L+LL=9uS 


(ON-€AN) *ZON*P=E7dNIN GN‘ L=9N O06 OG 
(aN-ZGN) *P=ZdN'IN LGN=440N 
2 L=aSVaGN 
HH HHH HH HH HH HH HE HH HE HH HH HH HH HH HHH HHH HHH HH HHH HHH HHH HH HH HHH HHH HHH HHH HHHHHH ©) (ON-€GN) *LGN*LL=€ZdNIN 
i LGN*0L=ZdN'IN 
AINGOW AYAVAM-LSOd LNIOd § AHL SLNAWSIdWI AGOD SNIMOTIO’N AHL | 2 
; ay HH HE HE HE HE HH IH HH HH HHH HE HHH HH HH HH HH HH HH HHH HH HHH HHH HHH HHH HHH HHH HHH HK HHHHKKHRKE B: 
HHH HH HE HE HH HE HE HH HE HE HH EH HH HH HE HHH HH HH HHH HH HHH HHH HHH HHH HHH HHH HHH HHHHHHHHHHHE @) ee 
2 AINGOW JAVAM-LSOd LNIOd 6 AHL SLNAWAIdWI 3GOD ONIMOTION FHL 2 
008 OL OD (%°AN’VN)dAI S 
NYN LAY (L°OR' VN) AL OOP HH HHH HH HH HHH HHH HH HHH HH HHH HH HH HH HH HHH HHH HH HH HH HHH HHH HHH HH HH HHH HH HHH KHKR* 2) 
€Z7TdN IN+ASVAN=ASVEN 0v6 ) 
ZdN IN+ ASVEN=aASVEN 0€6 00% OL OD (6°4N‘AN) AI 006 
L+HSVEN=aSVGN OL6 €7TdN IN+ASVEN=ASVEN OVE 
8US=(8uN) US ZdN IN+ASVEN=ASVEN O€E 
Lus=(LUN)us L+4SVEN=4SVEN OLE 
7TuS=(ZUN) US cUS=(ZUN) US 
GL-PL=(Z7UN)IS (ZUN)UYS-LL=(ZUN)IS 
GL+P7L=(LYN)IS (ZUN)YS+LL=(LUN)IS 
8L-LL=(SUN)IS (LUN) IS+(ASVEN)IS=LL 
8L+LL=(~uN)Is (ZUN) IS+LL=Zus 
TL-LL=(8uN)IS (ZTUN)IS-LL=(LUN) Us 
CL+LL=(LUN)IS ( LUN) US+ (ASVEN) US=LL 
(6UN) US+ (LUN) US+ (ZUN) US=7L JION+ LUN=ZUN 
(6UN) US- (SUN) US+ (ZUN) US=SL JION+ASVAGN= LUN 
(8UN)US- (LUN) US- (ZUN) US=8L LON‘ L=ZN OLE Oa 
9L=(9UN)IS ONS L=9N OEE OG 
(9UN) IS+ (PUN) IS+LL=LL GN‘ L=SN OVE Od 
(SUN) IS-(PUN)IS-LL=LL LGN=ad40N 
(OYUN) IS-(SUN) IS+LL=FL L=aSVaN 
(OLUN)US+EL=(€UN)IS (ON-€GN) * LGN*€=€7dN IN 
(OLUN) YS-EL=9L LON*Z=CdN'IN 
(€UN)IS+(€UN) IS+ (ASVAN) IS=(ASVEN)IS 5 
(LUN) IS+ (ASVEN) IS=LL HH HE HE HE HE HH HE HE HE HE HE HE HE HE HE HE HE HE HHH HH HH HH HHH HHH HH HHH HHH HH HH HHH HHH HEH HHH HHH HHH HHH HHH eh 
(€UYN)IS-(ASVAN) IS=€L 5 
GL+P7L=Zus SAINGOW AAVAM-LSOd LNIOd € AHL SLNAWHIdWI AGOD SNIMOTION AHL 3 
GL-7L=Lus 3 
SL+LL=(SHUN)US HEHEHE HE HE HEHE HE HE HE HE IE IE IE FE IE IE IE HE FE HE HE EH HH HHH HK HH HH HHH HHH HH HHH HHH HHH KE HK HK HHH HHH HH HHH ay 
8L-LL=(7uUN) Us » 
TL+LL=8us 006 OL OD (€°4UN'AN) AI 
CL-LL=( LYN) Us OOF OL OD (L°OR'AN) AI OO€ 
(6UN)IS+ (LUN) IS+(ZUN)IS=ZL 7 dN'IN+3SVaN=aSSVaNn OVL 
(6UN) IS-(8UN) IS+(ZUN) IS=SL L+ASVaEN=aSVEN OLL 
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(8)L-(9)L=(P7LUN) US 

(8) L+(9)L=OLuUS 
(L)L-(¢)L=9uS 
(L)L+(¢)L=(ZUN) US 

(€ LUN) IS+(9LUN) IS-=(PL)L 
(9OLUN)IS-(6UN)IS-=(OL)L 
(LLYN) US-( LLYN) YS=(7ZL)L 
(LLUN)UYS-(GLUN)US=(9L)L 
(TLUN) IS-(OLUN) IS=(LL) ao 
(TLUN)IS-(OLUN)IS-=(E€L)L 
(PLUN) YS-(8UN) US=(SGL)L 
(VLUN)US+(8UN)US=(6)L 
(LUN) US+ (9UN) IS-=(L4)L 
(LUN)US-(9YN)IS-=(8)L 
(SUN) IS-( PUN) YS=(S)L 
(SUN) IS+ (PUN) US=(9)L 
(€UN) IS-(ZUN)YS=(€)L 
(€UN)IS+(ZUN)US=(7)L 
(ASVAN) US+ (LUN) US= (ASVAN) US 
(LUN)US- (ASVAN) YS=(Z)L 
L+9LUN=LLUN 

L+GLUN=9LUN 

L+PLUN=S LUN 

L+€LUN=PLUNn 

L+ZLUN=€ LUN 

L+LLUN=ZLUN 

L+OLUN=LLUN 

L+6UN=0 LUN 

L+SuUN=6uUN 

L+LUN=8uNn 

L+9UN=LUN 

L+SUN=9UN 

L+PUN=SUN 

L+€UuN=PuUNn 

L+ZUN=CUN 

L+LUN=ZuUNn 

L+4SVEN= LUN 

aN‘ L=ZN OZ9L OG 

ONS L=EN O€9L OG 

GN‘ L=9N OPO9L OG 

L=aSVaGN 

(ON-€GN) *ZON*8L=€7dN'IN 
(€aN-ZAN) *8L=ZdN'IN 


HHH HHH HH HH HHH HHH HHH HHH HHH HHH HHH HHH HHH HHH HEHEHE HEHEHE HHH HHH HEHEHE KEHHHHHKHEEE 


Q2INGOW AAVEM-LSOd LNIOd 9L AHL SLNAWA'IGNWI AGOO ONIMOTION AHL 


HHH HHH HHH HHH HHH HHH HHH HHH HHH HHH HHH HEHEHE HEHEHE HHH KEKE HEHEHE KKH HHH HEHEHE HHH HHH 


NYOLAY (9L°AN°VN) AI 
€ZTdN IN+HSVEN=ASVEN 
7 dN IN+ dSVEN=ASVEN 
8+HSVAEN=SSVEN 
9US=(9UN) US 
GUS=(SUN) us 
€uS=(€UuN) us 
LL-GL=(LUN)IS 
LL+GL=(SUN)IS 
9L-PL=(€UN)IS 
Q9L+PL=(LYN)IS 
LL=(PvuN)Is 

(LUN) IS-(9UN) HS=LL 
(LUN) IS+ (9UN) YS=9L 
€L=(9UN)IS 


Oo0 00 0 0 


OO09L 
0v8 
0€8 
078 


(SUN)US- (PUN) IS=SL 
(SUN) US+ (PUN) IS=FL 
(€UN)US+(ZUN)IS=(ZUN)IS 
(€UN) US-(ZUN) IS=EL 
(LUN) IS+(ASVEN)IS=(ASVEN)IS 
(LUN) IS-(ASVEN) IS=LL 
LL+GL=(LUN)uUs 

LL-S L=Sus 

9L-PL=€us 

9L+P7L=(LUN)US 

LL= (pun) us 
(9UN) IS+ (LUN) US=LL 
(9UN)IS-( LUN) US=9L 
(SUN) IS+ (PUN) US=SL 
(SUN) IS-(P7UN)US=PL 
(€UN) IS-(ZUN) US=(ZUN) US 
(€UN)IS+(ZUN)US=9uS 
(LUN) US+ (ASVEN) US= (ASVAN) US 
(LUN)US- (ASVEN) US=LL 
L+9UN=LUN 

L+SGUN=9UN 

L+PUN=SUN 

L+€UN=PpuUn 

L+ZUN=€UN 

L+LUN=ZuUn 

L+4SVaGN=LUNn 

aN‘ L=ZN 028 Od 

ONS L=EN O€8 Od 

GN‘ L=PN 078 Od 

L=aSVaN 

(ON-€AN) *ZGN*8=€7dN'IN 
(aN-ZAN) *8=7dN'IN 


HHH HH HHH HHH HHH HH HHH HH HHH HHH HEH HHH HHH HHH HHH HHH HHH HH HEHEHE HHH HHHHHHHHHHHE 


ATNGOW AAVEAM-LSOd LNIOd 8 FHL SLNAWATIGNWI YACOO ONIMOTION AHL 


HHH HHH HHH HHH HH HH HHH HEHEHE HEH HHH HHH EHH HEHEHE H HEHEHE HHH HHHHHEHHHHEHHHHHHHHHHEH HE 


00O9L OL OD (8°4N°VN) AI 
€ZTdN IN+ASVAEN=ASVEN 
ZdN IN+ ASVEN=ASVEN 
P+HSVEN=aSVEN 
€UL+ZIL=(€uUN)IS 
€uUL-ZIL=(lLUN)IS 
LIL-OIL=(ZUN)IS 
LIL+OIL=(aSVEN)IS 
(ZUN) IS-(ASVEN) IS=ZIL 
(ZUN) IS+(ASVEN) IS=OIL 
€LL-ZUL=(€UN) US 
CIL+ZUL=(LUN) US 
LUL-OUL= (ZN) us 
LUL+OUL= (ASVAEN) US 
(€UN) IS-( LUN) IS=€1L 
(€UN)IS+( LUN) IS=LIL 
(€UN) US-( LUN) US=ECUL 
(€UN)US+ (LUN) AS=LUL 
(ZUN) US-(ASVAN) US=ZUL 
(ZUN) US+ (ASVEN) US=OUL 
L+ZUN=€uNn 

L+LUN=ZuUN 

L+HSVEN=LUNn 

aN‘ L=ZN OZP Od 

ON‘ L=EN O€F OC 

GN‘ L=9N OVP OG 


OOUvO O00 


008 
OV? 
Of? 
OCP 
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aNd 

NYNLAA 

€7dN IN+ASVAN=ASVAN 
7dN IN+HSVEN=aSVAN 
8L+aSVEN=aSVEN 


OvV9L 
O€9L 
Oc9OL 


€LUYS=(E€LUNjJUS 
ZTLUS=(ZTLUN)US 
LLY¥S=(LLUN) Us 
OLUYS=(OLUN)US 
6uUS=(6UN) US 
9uUS=(9UN)US 
GUS=(SUN) Us 
€uSs=(€UuN)uUs 

(7) L=(@LYUN)IS 
(€)L=(PUN)IS 

(7) L=(8uUN)IS 
(Z)0-(S$)O=(LLUN)IS 
(S$)O0+(Z)O=(€LUN)IS 
(L)0-(7)0=(€UN)IS 
(7)O+(L)O=(SUN)IS 
(9)0-(8)0=(SLUN)IS 
(9)0+(8)0=(6UN)IS 
(€)0-(24)0=(LUN)IS 
(L)0+(€)0=(LUN)IS 
(9L)L-(€L)L=(9)0 

(OL) L+(€L)L=(€)O 
(VL)L-(SL)L=(S)0 

(SL) L+(7L)L=(7)0 

(TL) L-(LL) L=(2)0 

(ZL) L+(LL)L=(L)0 

(OL) L-(6)L=(8)0 

(OL) L+(6)L=(L)0 
(8)L-(9)L=(PLUN)IS 

(8) L+(9)L=(OLUN)IS 
(L)L-(S$)L=(94N)IS 
(L)L+(S$)L=(Z@UN)IS 
(OLUN)US+(6UN)US=(0L)L 
(LLYN) IS-( LLYN) IS=(Z7L) aL 
(LLYN) IS-(SLUN)IS=(9L)L 
(OLUN) YS-(ZLUN) US=(LL)L 
(TLUN)US+(OLUN)UYS=(EL)L 
(PLUN) IS-(8UN) IS=(SL)L 
(7VLUN)IS+(8UN)IS=(6)L 
(LUN) IS+(9YN)US=(L)L 
(LUN) IS-(9UN)US=(8)L 
(GUN) US+ (PUN) IS=(¢G)L 
(GUN) US-( PUN) IS=(9)L 
(€UN) US+ (ZUN) IS=(€)L 
(€UN)US-(ZUN) IS=(7)L 
(ASVGN) IS+ (LUN) IS=(ASVAEN)IS 
(LUN) IS-(S3SVEN)IS=(Z)L 
(7) L=ZLus 

(€)L=(PUN) Us 

(7) L=(8uUN) US 
(7)O0-(S)O=LLUS 
(S$)0+(Z)O=ELUS 
(L)O-(7)O=EUuS 
(7)0+(L)O0=SUuS 
(9)0-(8)0=(SLUN) US 
(9)0+(8)0=6uS 
(€)O0-(L)0=(LUN)US 
(L)0+(€)0=(LUN)US 
(9L)L-(€L)L=(9)0 

(OL) L+(€L)L=(€)O 
(VL)L-(SL)L=(S)0 

(SL) L+(P7L)L=(7)O 

(TL) L-(LL) L=(2Z)0 

(ZL) L+(LL)L=(L)0 

(OL) L-(6)L=(8)0 

(OL) L+(6)L=(2)0 
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Time-Efficient Radix-4 Fast Fourier Transform 


L. Robert Morris 


Department of Systems Engineering and Computing Science 
Carleton University 
Ottawa, Canada 


1. Purpose 


This program uses the autogen technique [1-3] to implement a time efficient radix-4 fast Fourier 
transform. It is designed for those applications in which a fixed-size transform is performed repeatedly, 
such as speech analysis/synthesis (e.g., vocoders, spectrogram production) and two-dimensional picture 
processing. In such applications, speed optimization is especially profitable. Sample benchmark times 
for a 1K complex transform are: 


Computer Compiler Time 
DEC PDP-11/55 FORTRAN IV PLUS = _. 250 msec 
DEC VAX 11/780 FORTRANIV PLUS ~— 150 msec 


IBM 370/168 FORTRAN H 32 msec 


Finally, the program was derived from the radix-4 subset of Singleton’s mixed-radix FFT [4] and thus 
illustrates the application of autogen techniques to speed optimization of other DSP software, such as 
that presented in this book. 


2. Method — 


An autogen program is time efficient because most data-independent computation required for algo- 
rithm implementation is done prior to program execution. Such computation includes that necessary 
for data or coefficient access and algorithm flow control, including selection of the time optimum com- 
putational kernel (ck) applicable at any point in the algorithm. Conceptually, an autogen program for a 
selected size transform can be produced by unwinding the loops of any program whose flow is data- 
independent. The result is a customized program consisting essentially of a linear sequence of ck’s. 
Each ck may then be the optimum for that point in the algorithm and may contain precomputed con- 
stants which are pointers to data and/or coefficients necessary for execution of that ck. This code gen- 
eration can in fact be automated by simply transforming an existing program into a generator program 
by replacing all sequences which operate on data points with "write" statements. When executed, the 
new program will then generate code, into a disc-based file for example. The file contents are then 
compiled/assembled to produce a customized, time efficient program [1]. However, excepting those 
cases when the ck size is "small" [2], in-line code is prohibitive insofar as memory storage requirements 
are concerned, and is a retrograde step for software speed enhancement on machines with cache 
memories. An alternative program structure for autogen software is "threaded code" [2,5]. In threaded 
code, the memory storage is divided into 


(a) aset of computational kernels 
(b) a data-point storage space 
(c) a coefficient storage space (when necessary), and 


(d) a driver array, or "thread". 
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The thread elements are accessed sequentially and consist essentially of precomputed pointers to ck’s, 
data, and coefficients. This means that no matter how complex the algorithm-dependent computation 
required at program generation time to determine the time optimum ck to invoke at any point during 
algorithm execution, the relevant ck can be invoked at run time by a single "computed go to" based on 
the value of a precomputed array element. Alternatively, using the premise that each ck must accept at 
least one nonzero data pointer per invokation, a time efficient "branch on nonzero (integer) pointer" 
can re-invoke the current ck. Because DSP ck’s are generally executed a number of times before tran- 
sition to the next, this strategy -- "knotted code" [2] -- is to be preferred and is in fact used in this FFT 
autogen program. , : 


In the radix-4 FFT, each butterfly includes three complex multiplications. Examination of the algo- 
rithm reveals that the multiplications required can be minimized by implementation of 5 different 
butterfly ck modules [6]. Further, by performing each nontrivial complex multiplication using a 3 mul- 
tiply/3 add algorithm, possible when a table of (sin 6; — cos 0;), —(sin 6; + cos 6,) and cos 6; is stored, 
the 5 ck’s required have the following characteristics for an N = 4**M point complex transform: 


ck # Multiplications # Invokations 

1 0 [4**M—1]/3 

2 4 [4**(M—1)—1]/3 

3,4 8 2[4** (M—1)—1]/3 

5 9 4**(M—1)(M—7/3]+4/3 


The total number of butterfly invokations is then M[4**(M—1)] = MN/4. These require 7,856 real 
multiplications and 28,341 real additions for M = 5, compared to 28,920 real additions and 15,112 real 
multiplications for Singleton’s program. Using the DEC FORTRAN IV PLUS compiler, 59,725 
floating-point register load/stores were executed (for M = 5) compared to 77,763 for Singleton’s pro- 
gram. 


In the documented program, thread and trig table generation are done during a first pass. This 
simplifies the program in terms of transportability (no disc file manipulation required) but means that 
the generator program is resident during FFT execution. Program storage requirements, in 16-bit 
words, are approximately 0.6NM + 5.2N + ck storage space. For the IBM 370 FORTRAN H and DEC 
FORTRAN IV PLUS compilers, ck storage space is about 1100 and 1500 16-bit words, respectively. 
An indication of the program time efficiency is that during execution of the program on a PDP-11/55, 
only about 10% of time is devoted to operations other than load/store and add/subtract/multiply of 
floating-point operands [6]. The relative speed increase as compared to conventional programs is very 
machine dependent. Time savings due to reduction of algorithm-dependent (and thus data- 
independent) operations will be greatest on modern computers which have hardware floating-point pro- 
cessors such that floating-point instructions require the same order of execution time as the 
nonfloating-point instructions eliminated. On machines with slow "software" floating-point arithmetic 
instructions, the nonfloating-point operations are, relatively, of less importance. However, on these 
machines, the reduction in "expensive" data-dependent operations via the ability of the autogen tech- 
nique to economically invoke the time optimum ck (i.e., the one containing the fewest possible data- 
dependent operations), provides valuable time savings. 


As noted earlier, the program was derived from the radix-4 subset of the well known Singleton 
mixed-radix program and implements 


N-1 _;2 ng 
X(k) = ¥ x(ne — , k=0,...,.N—1 (forward transform) 
n=0 
or 


N-1 jot ok 
x(n) = (/N) ¥ X(kKe NV, n=0,...,N—1 (inverse transform) 
k=0 
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The following table summarizes relative performance on the PDP-11/55. For completeness, execu- 
tion times for an assembly language floating-point autogen FFT have been included [7]. As noted in 
[6], use of floating-point registers as temporary storage locations can be better optimized at the assem- 
bler level, thus reducing the load/store operations of floating-point data. 


Autogen Autogen Singleton 
N (a) Assembler (b) FIVPLUS (ce) FIVPLUS) (JM@ = (Ob) 
16 0.90 msec 1.58 msec 3.23 msec fe 2.04 
64 5.80 msec 9.07 msec 14.56 msec dal 1.60 
256 33.00 msec 49.57 msec 71.51 msec 2.16 1.44 


1024 168.70 msec 250.01 msec 343.88 msec 2.03 1.37 
As per sec. 2, the proportion of time-favorable butterflies decreases with increasing N. Thus, use of this 
program for transforms of size 4 K or more is inappropriate from a time/space tradeoff point of view. 


3. Program Description 


3.1 Usage 


The program consists of the basic subroutine RADIX4, which includes the 5 butterfly ck’s and the 
autogen code, and a small subroutine RAD4SB, used by the autogen. Much of the notation of the ori- 
ginal Singleton program has been preserved. The user passes the data in an array "A", in labeled com- 
mon block "AA". This step is essential for ensuring "optimum" data addressing techniques used by optimizing 
compiler generated code. 


3.2 Description of Parameters 


The basic call is of the form 
CALL RADIX4(M,IFLAG,JFLAG) 


where N = 4**M complex points are to be transformed and 2 < M < 5 


IFLAG = | on the first call 
= (0 on subsequent calls with the same value of M. 


JFLAG = —1 for a forward transform 
= +1 for an inverse transform. 


On input, the array A (passed in labeled common) contains the complex data points, real in odd array 
components, imaginary in even array components. On output, the array A contains the complex DFT 
coefficients X(k) or x(n) as per JFLAG. 


3.3 Dimension Requirements 


The DIMENSION statements in the main program and both subroutines can be modified to reflect 
the largest size transform to be implemented as follows: 
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Array Dimensions 
A IX 2 


M=2 32 38 27 
M s 3 128 144 135 
M<4 $12 658 567 
mS > 208% 2996 2295 


Arrays A and T are real (floating-point) and array IX is INTEGER%2, i.e., 16-bit. 


3.4 Summary of User Requirements 


On each call to RADIX4, load A with the input data and retrieve the output data from A. On the 
first call for a given size M, use IFLAG=1, on subsequent calls use IFLAG=0. For a DFT use 
JFLAG = —1, for an IDFT use JFLAG = +1. 


4. Test Program 


For 3 < M <5, N = 4**M, compute the DFT of a complex sequence generated by UNI, then 
compute the IDFT of the result. The RMS error between DFT input and IDFT output is printed in 
each case. For N = 64, the DFT/IDFT input and output are listed. Note that the error will vary 
according to the format (e.g., bits for fraction) of floating-point numbers on any particular machine. 


[1] 


[2] 


[3] 


[4] 


[5] 
[6] 


[7] 
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Table 1 
DFT INPUT DFT 

REAL IMAG REAL 

. 229256 0.766875 31.377766 
683177 0.509191 -1.565459 
.874560 0.644641 -0 .562676 
.847468 0.353963 -1.280178 
. 398892 0.457094 -3.942028 
. 236309 0, 143182. -=2, 127302 
.166052 0.226027 -0.364557 
662459 6.250212. «6.799656 
.617697 0.262465 3.244458 
.512668 0.939207 1.399128 
.624028 0.422382 0.541036 
. 939706 0.282068 -0.624628 
469218 0.054079... =45 387045 
.519831 0.396827 -1.564005 
.113157 0.607517 -0.138395 
. 701507 0.887055 -2.695570 
. 331433 0.220961 -1.135619 
. 168616 0.992249 2.194667 
. 367070 0.497800 -0.264126 
254104 0: 797795... .—-4, 4192945 
.133912 0.116143 2.231800 
. 212882 0.783360 1.686178 
982931 0.788486 -3.398292 
. 365426 0.476909 0.976109 
.827320 0.102528 -0.999293 
. 943264 0.684253 -3.917623 
.729065 0.722039 1.067974 
. 340034 0.410316 -4.502048 
. 384621 0.961848 0.645658 
.480094 0.096515 -1.795084 
.519834 0.761804 0.052220 
. 704444 0.340203 2.705629 
.460761 0.475524 -1.924339 
. 748171 0.654245 -1.059475 
.421876 0.441382 0.055644 
774413 0.625799 6.071876 
524052 0.092862 -4 .694662 
. 984750 0.829579 2.227794 
.192729 0.291600 1.329456 
. 385426 0.467851 1.600311 
. 320288 0.242745 -0.051197 
. 801388 0.025701 -0.497059 
.985884 0.904574 -0.116100 
.105523 0.237869 2077721 
. 253360 0.134264 1.047104 
. 979304 0.629147 -3.381618 
.445763 0.913431 0.277854 
270815 0.408028 2.875121 
.234614 0.644116 -2.133524 
. 723286 0.768105 -0.886670 
194135 0.632751 1.965688 
158584 0.594914 0.325368 
.158079 0.353140 -0.403606 
055275 0.430064 -0.313278 
546173 0.499216 -4.102506 
.915547 0.623864 -1.932684 
.633616 0.045868 0.861630 
.580915 0.172191 0.113929 
.969972 0.916135 1.158827 
.400834 0.777754 0.789765 
568952 0.427517 -~-2.839772 
.124854 0.777654 -0.988682 
.047415 0.260892 4.481467 
. 069980 0.787919 -1.661487 


OUTPUT 


32 


=O. 
.274236 
.779767 
.433803 
.555010 
.019432 
.879471 
.330645 
.712091 
.833928 
. 266858 
.579592 
. 370283 
.839718 
.165845 
.030361 
. 753474 
.562021 
3.509271 . 
.938169 

.474952 
.256503 
.695807 
.800101 
. 189232 
.777765 
.440295 
.914912 
. 368239 
.359995 
. 261653 
.254483 
224388 
.737301 
1371 
.357120 
. 556963 
.476440 
.237184 
.647760 
. 466700 
.350873 
. 162251 
.395429 
.410234 
.371036 
.404488 
313334 
. 253150 
.067677 


= 


IMAG 


.033501 


613354 


gn 280138 
1.931254 


oOO- - OP WWD - = 


. 078680 
-050208 
- 410821 
-831643 
. 341182 
.071988 
. 218662 
-910510 
- 583505 
- 320888 
- 184884 
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IDFT OUTPUT 


Table 1 
(Continued) 
DFT INPUT 

REAL IMAG REAL 
.229256 0.766875 0.229256 
.683177 0.509191 0.683177 
.874560 0.644641 0.874560 
847468 0.353963 0.847468 
.398892 0.457094 0.398892 
. 236309 6°433182 0.236309 
.166052 0.226027 0.166052 
662459 0.250212 0.662459 
.617697 0.262465 0.617697 
.512668 0.939207 0.512668 
.624028 0.422382 0.624028 
. 939706 0.282068 0.939706 
469218 0.054879 0.469218 
.519831 0.396827 0.519831 
329457 0.607517 0.113157 
701507 0.887055 0.701507 
.331433 0.220961 0.331433 
168616 0.992249 0.168616 
.367070 0.497800 0.367070 
. 254104 0.797795 0.254104 
.133912 0.116143 0.133912 
. 212882 0.783360 0.212882 
.982931 0.788486 0.982931 
. 365426 0.476909 0.365426 
.827320 0.102528 0.827320 
943264 0.684253 0.943264 
.729065 0.722039 0.729065 
. 340034 0.410316 0.340034 
. 384621 0.961848 0.384622 
.480094 0.096515 0.480094 
.519834 0.761804 0.519834 
704444 0.340203 0.704444 
.460761 0.475524 0.460761 
748171 0.654245 0.748171 
.421876 0.441382 0.421876 
.774413 0.625799 0.774412 
.524052 0.092862 0.524052 
.984750 0.829579 0.984750 
.192729 0.291600 0.192729 
. 385426 0.467851 0.385426 
.320288 0.242745 0.320288 
. 801388 0.025701 0.801388 
.985884 0.904574 0.985884 
.105523 0.237869 0.105523 
.253360 0.134264 0.253360 
.979304 0.629147 0.979304 
445763 0.913431 0.445763 
. 270815 0.408028 0.270815 
.234614 0.644116 0.234614 
. 723286 0.768105 0.723286 
.194135 0.632751 0.194135 
.158584 0.594914 0.158584 
.158079 0.353140 0.158079 
.055275 0.430064 0.055275 
.546173 0.499216 0.546173 
915547 0.623864 0.915547 
.633616 0.045868 0.633616 
.580915 0.172191 0.580915 
969972 0.916135 0.969972 
.400834 0.777754 0.400834 
.568952 0.427517 0.568952 
124854 0.777654 0.124854 
047415 0.260892 0.047415 
. 069980 0.787919 0.069980 
RMS ERROR FOR M = 3 IS 

RMS ERROR FOR M = 4 IS 

RMS ERROR FOR M = 5 IS 


CODCOD COC OOOO COO OO CO OOO OOOO OOO OOOO OOOOOOOOOOOOCOOOCAOCCOCOAOGAGCCOCCCOCoCoO°o°o eo 


IMAG 


.766875 
U9 191 
.644641 
<ao3903 
.457094 
133182 
.226027 
pe oUZt2 
.262465 
93920 / 
.422382 
. 282068 
.054879 
. 396827 
SUS? 
;» 86/055 
5220967 
.992249 
-497800 
pverTso 
.116143 
. 783360 
. 788486 
.476909 
5402528 
.684253 
-7e2039 
.410316 
.961848 
. 096515 
.761804 
. 340203 
.475524 
.654246 
.441382 
.6e5799 
.092862 
«8629579 
.291600 
-467851 
.242745 
.025701 
.904574 
$25 7869 
.134263 
.629147 
.913431 
-408028 
.644116 
. 768105 
632751 
.594914 
.353140 
.430064 
-499216 
.623864 
.045868 
rie? 
“916135 
~TITITO4 
-427517 
. 777654 
- 260892 
Toros 
0.114103E-06 

0.221640E-06 

0.566590E-06 
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AIAUALLNA AIMILINW 8 
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Two-Dimensional Mixed Radix Mass Storage Fourier Transform 


Richard C. Singleton 


SRI International 
Menlo Park, CA 94025 


1. Purpose 


Two-dimensional mixed radix fast Fourier transform algorithms are presented, for use with a mass 
storage data file. These transforms operate on both real and complex input sequences. A transform (or 
inverse) is computed in two passes of the mass storage file. The method can be applied to any multi- 
dimensional transform that operates on the dimensions one by one, and is not limited to two dimen- 
sions. 


Transforms of size 1024 by 1024 have been computed on the Burroughs 6700 and the CDC 6400; 
on both machines the program runs without problem in the normal mix, and the transform size is 
apparently limited only by the size of the mass store. 


2. Method 


The basic fast Fourier transform subroutines used here (FFT and REALT) are presented and tested 
in an earlier Section (1.4.). The new idea here is a permutation method to allow use of the basic FFT 
routines to do larger transforms on data on mass store. 


If we assume a mass storage file to be an N1 by N2 array, stored with records written by columns, 
where N1 is an integer multiple of the record length, then the FFT by columns is simple. The problem 
is, how do we transform the rows? 


This problem is solved here by following the FFT operation on the columns by a permutation of the 
input buffer so that the intermediate results can be written back to the input file in such a way as to 
partition the file into M1 by M2 sub-matrices, where M1*M2 is the record size of the file. After this is 
done, the file can be read back either by rows or by columns. M1 must be a multiple of 2, as the data 
are complex. N1 must be a multiple of M1*M2, the record size. However N2 need only be a multiple 
of M2. 


On the first pass of the file, columns of length N1 are read into the input buffer in consecutive 
groups of M2. On the second pass of the file, N2/M2 sub-matrices of size M1 by M2 are read into the 
input buffer; the buffer then contains the values for M1 rows of length N2. Thus the input buffer must 
have dimension MAX0(N1*M2,N2*M1). 


For a real transform, 2*M2 additional buffer locations are needed for the folding frequency Fourier 
coefficients, thus the buffer must have dimension MAX0((N1+2)*M2,N2*M1). Also, during a real 
transform or inverse, a working store of size 2*N2 is set up within the subroutine FFT2T or FFT2I. 


This method can be generalized to higher-dimensional Fourier transforms, by sub-dividing either or 
both of the dimensions N1 and N2 used here. The method can also be applied to other transforms that 
Operate independently on rows and columns. 
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3. Usage 
3.1 Transform of Complex Input Data, N1/2 By N2 
CALL FFT2T(A,J1,K2,11,M2,0) (transform, no scaling) 
CALL FFT2I(A,J1,K2,11,M2,0) (inverse, scaled) 
where 
A Is real array dimensioned to MAX0(N1*M2,N2*M1) 
M1=I1*2 Is the column dimension of the sub-matrix partitioning 
M2 Is the row dimension of the sub-matrix partitioning 


N1=J1*M2*M1 Is the column dimension of the data 
N2=K2*M2 Is the row dimension of the data 


On entry, the complex data are on the mass storage file in columns of length N1, or N1/2 complex 
values. Real and imaginary components alternate. 


On completion of the transform, the data are left on the mass storage file, partitioned in sub- 
matricies of size M1 by M2, so that the results can be read either by columns or rows. The inverse 
transform expects input in this form. On completion of the inverse, the data are left on the mass 
storage file in sequential records by columns, scaled by 2/(N1*N2). 


A call on FFT2T, followed by a call on FFT2I with the same input parameters, produces an identity 
transformation. 


The mass storage file LUN should be declared by the user to have a logical record size of M1*M2; 
the unit number LUN is communicated through labeled common FFTCOM. 


3.2 Transform of Real Input Data, N1 by N2 


CALL FFT2T(A,J1,K2,11,M2,1) (transform, no scaling) 
or 
CALL FFT21(A,J1,K2,11,M2,1) (inverse, scaled by 1/N) 


where the parameters are the same as in 3.1, except that any nonzero value for the sixth parameter 
causes the additional steps for a real transform or inverse to be done. 


On completion of the transform, the Fourier coefficients are left on the mass storage file, partitioned 
in sub-matrices of dimension M1 by M2, with the cosine and sine coefficients alternating down 
columns. The N2 folding frequency cosine coefficients (alternating with zero sine coefficient values) 
are stored at the end of the main file in records of length M1*M2, with a possible final short record to 
bring the total length of the added records to 2*N2. The inverse transform expects data in this form. 
On completion of the inverse, the real data is in sequence down columns. 


A call on FFT2T, followed by a call on FFT2I with the same input parameters, produces an identity 
transformation. 


3.3 Other Subroutines Needed: 


TRNSP, EXCH and XFT --- included here 
FFT, FFTMX, REALT, ISTKGT and ISTKRL --- from Section 1.4 


MFREAD and MFWRIT -- machine-dependent mass storage read and write subroutines. The versions 
used here were copied from "Optimized Mass Storage FFT Program," by Donald Fraser. His 
subroutines for the PDP 11 work on the Burroughs B6700 computer. They should also work on 
an IBM 360/370. For publication, the mass storage file LUN has been replaced by an array 
RMAS; this version was also tested. 
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3.4 Error Conditions 


The signs of J1, K2, I1 and M2 are ignored. Thus the only error condition recognized by FFT2T or 
FFT2I is 


Ji*K2*I11*M2 = 0 


This condition results in an error condition and no transform. 


4. Testing 


The subroutines included here were tested on a Burroughs B6700 computer. A real transform of 
size Nl = 240 by N2 = 225 was done with FFT2T, using consecutive integers from 0 to 53,999 as test 
data. The inverse was then done with FFT2I, and the result compared with the input. The square root 
of the sum of squares of the errors divided by the sum of squares about the mean for the input was 
computed as an accuracy check; the result was 0.147E-10 (see Table 1). The record size used was 240, 
allowing sub-matrices of size M1 = 16 by M2 = 15. The buffer size was thus 3630, and an additional 
450 working storage locations were set up within FFT2T and FFT2I on a temporary basis, using 
ISTKGT and ISTKRL. Average CP time for the transform or inverse was 82 seconds. Most of this is 
time for FFT and REALT, as when the calls on these subroutines were. removed and the permutation 
alone was tested, the average time each direction was 7 seconds. 


Table 1 


TEST OF TWO-DIMENSIONAL MIXED RADIX FFT 


RECORD SIZE = 240 BUFFER SIZE = 3630 240 BY 225 TRANSFORM 
RMS ERROR NORM FOR TRANSFORM-INVERSE PAIR = .147E-10 
MAXIMUM STACK SIZE = 4147 
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CHAPTER 2 


Power Spectrum Analysis and Correlation 


L.-R. Rabiner 


Introduction 


The programs in this and the next section are various implementations of high speed convolution, 
correlation, and spectrum analysis algorithms. A wide variety of techniques have been proposed in this 
area and the four programs presented are only a small sampling of those available. However, they do 
serve to illustrate the types of techniques which have been applied in a wide variety of applications. 


The first program by Rabiner, Schafer, and Dlugos is an implementation of the method of modified 
periodograms, as proposed by Welch [1], for directly estimating the power spectrum of a signal. 


The second program, again by Rabiner, Schafer, and Dlugos, is an implementation of the correlation 
method, as proposed by Rader [2], for directly estimating the correlation function of a signal. 


The third program in this section, by Carter and Ferrie, is one for estimating the coherence function 
of a signal. Such techniques are widely applied in the areas of sonar, radar, and other underwater appli- 
cations of digital signal processing. 


A recent IEEE Press volume [3] describes several other valuable algorithms for power spectrum 
analysis. Although programs are not included in that volume the interested reader should consult that 
reference for alternative methods of computing spectra and correlation functions of digital signals. 
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1. Purpose 
This program can be used to obtain a direct estimate of the auto-power spectrum of a signal using 
the method of averaging periodograms. 


2. Method 


The use of the FFT in this manner was first described by Welch [1]. The essential features of the 
method are given below. Further details are available in Refs. [1-3]. 


A sequence x(n) has a power spectrum denoted P...(w). The modified periodogram spectrum esti- 
mate is obtained by dividing an available segment of x(n), 0 < n < N—I1 into K overlapping segments 
of length L. In this implementation, the segments overlap by L /2 samples, giving the total number of 
segments as 


K =[(N — L/2)/(L/2)] 
where [x] denotes the integer part of x. The / s segment of data is defined as 
x(n) = xGL/2+nwian) On << L-l, OS i < K-1 (1) 


where, w,(n) is an L-point data window (e.g., rectangular, Hamming, etc.). The M-point (M > L) 
DFTs of the windowed segments x;(”) 


M-1 es . 
X= SE xl(ne “ O<k<M-1, 0<i< K-l (2) 
n=0 


are computed using an FFT algorithm. (If L < M, the sequence x;(n) is augmented with M—L zero 
valued samples.) The modified periodograms 


Si] ai 6 cee MI, 0< 1 < K-1 (3) 


are averaged to produce the spectrum estimate at normalized radian frequency 27k/M 


K-1 
5. Cpe es P'S) 0 < ke <M. (4) 
KU i 
where 
L-1 
U= > win). (S) 
n=0 


The quantity U is necessary for the spectrum estimate to be unbiased [1]. 


It can be shown [1,3] that the expected value of S,,.(2a7k/M) is 


E(S.,Qrk/M)1 = ~~ { P..(0) W(2rk/M—8) do (6) 
Lt 5 
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The effective spectrum smoothing "window" is 


Wt) = 7 |W (cw) |? , (7) 
where 
pai 
W(w) = ¥& wi (nde lon (8) 
n=0 


is the Fourier transform of the data window sequence, w,(). 


It can also be shown [1] that the variance of the spectrum estimate, S,.(27k/M), is 


dds 
9K 


Since the width of the spectrum window W(e/®) is inversely proportional to L, and the variance of the 
spectrum estimate is inversely dependent on K, it is clear that for N = (K+1) (L/2) fixed, good fre- 
quency resolution and low variance are conflicting requirements. Clearly resolution can be traded for 
low variance by using many short segments, and greater resolution can be achieved at the expense of 
greater statistical variance of the estimate. 


var [S,,.2ak/M)] = P2(2ak/M) . (9) 


3. Program Description 


A flow chart of the program is shown in Fig. 1. 


READ 
ANALYSIS 
PARAME TERS 


M, IWIN, L, N, FS 


ESTIMATE 
ME AN 


GENERATE 
AND STORE 


OBTAIN 2 
SEGMENTS 
OF DATA 


Fig. 1 Flow diagram of program, PMPSE. 


First the program reads in parameters such as FFT size, window type, window length, maximum 
number of data samples available and sampling frequency of the data. Then the mean is estimated so 
that it can be subtracted from the data. (This is desirable so that the low frequency portion of the spec- 
trum is not obscured.) Next the modified periodograms are computed for the available data, and accu- 
mulated according to Eq. (4). 


A notable feature of the program is that the periodograms |X,(k)|? are computed two at a time. It 
can be shown [1] that if two real segments x,(m) and x,,,(”) are used as real and imaginary parts of a 
sequence 
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x(n) = x(n) + jxia)(n) (10) 
then 
X(k)X*(k) + X(M—k)X*(M—k) = 2X (k) 2? + |X, 7] Ok < M/2 (11) 


where X(k) is the M-point transform of the complex sequence, x(n), and se and X,,,(k) are the 
M-point DFT’s of x,(n) and x,,,(m) respectively. Thus the left hand side of Eq. (11) gives two 
periodograms from one FFT computation. 


The final step in the program is the normalization by K-U and the computation of 
20 logiolS,,(27k/M)]. At this point the spectrum estimate is available for plotting, etc. 


An estimate of the autocovariance of x(n) is then obtained by taking the inverse DFT of the power 
spectral estimate. Since only a sampled version of the power spectrum is available, the resulting auto- 
covariance is an aliased version of the true autocovariance of x(n) [2]. 


3.1 Usage 


The main program (PMPSE) requires two user supplied subroutines; GETX for accessing blocks of 
data, and FFT, a complex, radix 2, FFT subroutine. The main program reads in analysis parameters, 
and then computes the estimates of the mean and power spectrum and then doés an inverse FFT to 
provide an autocovariance estimate of the input. The program prints out the log power spectrum (in 
dB), and the autocovariance function. 


Although no graphical output capability is provided with the program, such can be supplied by the 
user at the places indicated in the main program. Typical graphical output is shown in the examples 
given below. 


3.2 Description of Program Parameters 
The main program PMPSE reads the following parameters: 


M FFT length. M must be a power of 2 (for compatibility with the FFT subroutine) that is less 
than or equal to the maximum value MAXM specified in the main program. 


IWIN Type of window used on data. If IWIN=1, a rectangular window is used; if IWIN=2, a Ham- 
ming window is used. 


ie | Window length. Must be less than or equal to M. 


Maximum number of data samples available for use in the analysis. Because of the windowing 
applied to the data, it is preferrable that N be exactly divisible by L/2. Thus the program 
processes only NSECT = (N-—L/2)/(L/2) overlapping segments of length L or a total of NP = 
(NSECT +1)(L/2) samples. 


FS Sampling frequency in Hz. Used only for printing the power spectrum values. 


The user must supply the routine GETX which provides values of x(n) to the main program. The 
call to this routine is 
CALL GETX (X,NRD,SS) 
where the parameters X, NRD, and SS, are 


X Array in which the samples of x() are stored 
NRD_ Number of samples of x(7) to be read into array X. 


Ss Starting sample number of x(7), i.e., samples x(SS) to x(SS + NRD — 1) are read into array X 
in the subroutine GETX. Note that SS is a floating point number to allow for maximum flexi- 
bility. The first input sample is assumed to be x(1), i.e., SS = 1. 


The main program PMPSE, and test subroutine GETX are given in the Appendix. The subroutine 
FFT is a complex, radix 2, FFT subroutine with calling sequence CALL FFT(X,M,INV), where 
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M = Size of FFT (a power of 2) 
X = Complex array of length M 
INV = 0 for direct FFT 


1 for inverse FFT 


3.3 Dimension Requirements 


The real array XA is a data buffer and must be dimensioned to be the maximum window length; 
i.e., MAXM where MAXM is the largest FFT length. The complex array, X, (the working array for 
the FFT), and the real array, WD, (the window array) must be dimensioned MAXM. The real array 
SXX is used for accumulating the spectrum estimate. The real array XFR is used to generate a fre- 
quency scale corresponding to the DFT sample frequencies and the real array ILAG is used to generate 
a set of lag indices for printing. The arrays SXX, XFR and ILAG are all dimensioned (MAXM/2 + 
i: 


3.4 Summary of User Requirements 

(1) Provide the subroutine GETX which specifies the input data x(). 
(2) Provide the complex FFT subroutine FFT. 

(3) Specify the analysis parameters, M, IWIN, L, N, and FS. 


(4) Provide graphical output of the power spectrum and autocovariance estimates. 


4. Test Problems 


To illustrate the use of this program a simple test problem is provided in the form of the subroutine 
GETX which generates samples of the sequence 


x(n) = cos(27n/10) , : 
representing samples of a sinusoid of frequency FS/10, where FS is the sampling frequency. 
Example #1 


For this example, the parameters are M = 128, IWIN = 2, L = 64, N = 256, and FS = 10000. That 
is, a segment of length 256 is sectioned into 7 sections of length 64 each weighted with a Hamming 
window, and the 128-point periodograms are computed and averaged. Table 1 gives the numerical 
values of the power spectrum and autocovariance estimates for this example and Figure 2 is a plot of 


these values. 
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Fig. 2 Plots of the log power spectrum and the autocovariance for Example #1. 


This example should be compared to Example #1 of the program CMPSE of section 2.2 in this book. 
In that example a 63-point Hamming window was applied to the autocorrelation function. In this case a 
64-point Hamming window was applied to the data. According to Eq. (7) the spectral window for this 
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example is essentially the square of the spectral window for Example #1 of CMPSE. Thus the width of 
the main lob (between the first zeros) is almost the same but the side lobes are twice as low (in dB). A 
comparison of Fig. 2 of PMPSE and Fig. 3 of CMPSE confirms this assertion. 
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Fig. 3 Plots of the log power spectrum and the autocovariance for Example #2. 


Example #2 


For this example M = 128, IWIN = 1, L = 64, N = 256 and FS = 10000. That is, everything is the 
same as Example #1 except a rectangular window is used. The results are given in Table 2 and Fig. 3. 
Again, this example should be compared to Example #2 of CMPSE. 
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Fig. 4 Plots of the log power spectrum and the autocovariance for Example #3. 
Example #3 


It should be noted that the FFT length M can be greater than or equal to the window length L. 
Obviously the greatest efficiency results when M = L, however if a more detailed representation of the 
spectrum is desired, M can be larger than L as in Examples #1 and #2. In this example, M = L = 
128, IWIN = 2, N = 256, and FS = 10000. In this case, about half as many segments are obtained 
from the 256 samples of data as in Example #1 (implying greater variance for a random signal) but the 
resolution is twice as good. The results are shown in Table 3 and Fig. 4. 
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Table 2 


M= 128 NP= 256 L= 64 SAMPLING FREQUENCY=10000.0000 
XMEAN= .69337E-06 
LOG POWER SPECTRUM 
FREQ DB FREQ DB FREQ DB FREQ DB 
.000 -23.4182 78.125 -41.2622 156.250 -22.7620 234.375 -40.0745 
312.500 -20.7564 390.625 -37.5837 = 468.750 -17.2393 546.875 -33.4572 
625.000--11.7172. 703.725 =25.1759 ~ 781.250. -2.5602 859.375 -14.1902 
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Table 3 
WINDOW TYPE=HAMG 
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XMEAN= .69337E-06 
LOG POWER SPECTRUM 
FREQ DB FREQ DB FREQ DB FREQ DB 
.000 -80.3770 78.125 =80 .1718.-.156.250 -—79-.5365 234.375 -78.5186 
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1. Purpose 


This program can be used to provide an estimate of either the autocorrelation function of a signal, 
or the cross-correlation between two signals using FFT techniques. An estimate of the power spectrum 
of the signal (obtained by computing the discrete Fourier transform of a windowed version of the corre- 
lation function) is also obtained in the program. 


1.1 Method 


The program is an implementation of the procedure originally described by Rader [1]. Figure 1 
shows a block diagram of the signal 


x (nN) USE FFT T 
COMPUTE» Ryy (m), Os MsL-t Sxy (kK) 
y (Nn) ESTIMATES OF (x) FFT 
CORRELATION FUNCTION 


w (m) 


WINDOW USED TO 
REDUCE EFFECTS 
OF USING A | 
FINITE LENGTH 
RECORD 


Fig. 1 Block diagram of the correlation method of power spectrum analysis. 


processing within the program. In the general case there are two input signals, x(n) and y(n), which 
are defined for the interval 0 < n < N — 1. Outside this interval it is assumed that x(n) and y(n) are 
0. We wish to compute the correlation function 


N-1l-m 
Rylm) =P (e(n)-O(n+m)-7) OS m<L-1, (1) 
n=0 
where xand jare the estimated means of x(n) and y(n), i.e., 

et 
= x x(n) (2a) 

1 N-1 
y=— DY yn). (2b) 

N n=0 


The program offers the option of subtracting the estimated means or not. If the means are subtracted 
an estimated covariance function is obtained. If the means are not subtracted, an estimated correlation 
function is obtained. For simplicity we shall use the term correlation function for both cases unless this 
would lead to confusion. Since, in general, N >> L, i.e., the number of samples of the signal is large 
compared to the number of correlation lags of interest, an efficient procedure for implementing Eq. (1) 
is to use FFT methods to accumulate partial correlation sums (being careful to avoid the aliasing effects 
of the circular correlation from the FFT), and then inverse transform the result to give the desired 
correlation function. Details of this procedure are given in Refs. [1-3]. It should be noted that for this 
method if the size of the FFT is M points, then M/2+1 values of the correlation function are obtained. 
Thus, the FFT size must be at least twice the desired number of correlation values, L. 
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To obtain the power spectrum estimate from the correlation function one of two procedures is used. 
If y(n) = x(n), i.e., the correlation is an autocorrelation (autocovariance), the windowed autocorrela- 
tion function, R,,.(m), is created as 


R,,.(m) = R,.(m)w(m) OS in Sy Hy 
= R,.(M—m)w(M-m) M-L+1<m<M-1 
= 0 otherwise (3) 


where w(m) is a symmetric finite duration window which tapers R,.(m) to 0 as m tends to L — 1. 
Equation (3) shows that R,,(m) is a symmetric function so that the resulting DFT of the sequence of 
Eq. (3) is a real function which is a smoothed estimate of the power spectrum S,,(e/”). 


If y(n) # x(n), ie., the correlation is a cross-correlation (cross-covariance), the signal R,,(m) is 
created as 


a 


Ry = Ry(m)w(m) O<m<L—-I 
= 0 | otherwise (4) 


where w(m) again represents a finite duration window. In general w(m) is a rectangular window since 
the "origin" of R,,(m) need not be at m = 0; thus arbitrarily tapering R,,(m) to 0 can lead to errone- 
ous estimates of the cross power spectrum. [For further discussion of this important issue see Ref. Z. 
p. 409-410]. The cross power spectrum estimate is again obtained as the DFT of the sequence of Eq. 


(4). 


2. Program Description 


The program flow is shown in Fig. 2. 


READ IN 
ANALYSIS M, N, MODE, FS 
PARAMETERS 


COVARI~ 
ANCE? 
CALCULATE GENERATOR 
MEANS 
y (n) 
GENERATOR 


CALCULATE — 
CORRELATIONS y (n) 
GENERATOR 


WINDOW 
RESULTING IWIN, L, NFFT 
CORRELATION FUNCTION 


COMPUTE ESTIMATE 
OF POWER SPECTRUM 


Fig. 2 Flow diagram of program, CMPSE. 


First the program reads in the parameters for performing the correlation analysis, namely the section 
size (FFT size), M, the total number of samples used in the estimate, N, and the type of correlation 
desired, i.e., autocorrelation, cross-correlation, autocovariance, or cross-covariance (as determined by 
the variable MODE). If a covariance function is desired, the next Step is calculation of the means of 
x(n) (and possibly y()) using the user provided subroutine(s) GETX (and GETY) which fill buffers 
with a block of samples of x() (and y(n)) from a prescribed starting sample. (For simplicity we 
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assume the N samples of x(n) are numbered from 1 to N). Following computation of the means, the 
correlation function of Eq. (1) is determined by proper sectioning of x(n) into blocks of M samples 
which overlap by M/2 samples. The implementation of Eq. (1) is done by computing M-point FFT’s of 
the individual sections of x(n) (and y(n)), accumulating sums in the frequency domain (for computa- 
tional efficiency), and inverse transforming the final result back to the time domain. 


Following the correlation computation the program prints the first (M/2+1) correlation values. To 
obtain a power spectrum estimate, window parameters are then read in and the appropriate windowed 
correlation function (either Eq. (3) or Eq. (4)) is computed. The window parameters are window type 
(IWIN - either rectangular or Hamming), window duration (L), and size of FFT used to obtain the 
spectral estimate (NFFT). The program then prints out the log spectral estimate in dB. 


2.1 Usage 


The program consists of a main program (CMPSE), two user supplied subroutines (GETX -and 
GETY) for accessing blocks of data, and a complex FFT subroutine (FFT). The main program reads in 
the appropriate parameters for the analysis, and the program computes the correlation function and 
prints its values, and from this array it computes an estimate of the log power spectrum (dB), and 
prints its values. Since the FFT subroutine is a radix 2 complex FFT, the section size must be a power 
of 2. Although no graphical capability is provided with the program, it is suggested the user insert 
appropriate graphics displays at the points denoted in the program to facilitate use of the program. Typ- 
ical graphical output is shown in the examples with this program description. 


2.2 Description of Program Parameters 
The main program reads the following parameters: 


M Section size. M must be an integer that is a power of 2 (for compatibility with the FFT subrou- 

| tine) and M must be less than or equal to a maximum section size MAXM as specified in the 
main program. (For this program MAXM is set to 512). Note that M/2+1 must be greater 
than or equal to the total number of lags desired. 


N Total number of samples of x(n) (and y(n)) used in the analysis. It is assumed that samples 
of x(n) and y(n) are numbered from 1 to N. (Note: For computers with 16-bit integer word 
lengths, N must be less than 32768.) 


MODE Parameter which determines the type of correlation to be computed. 


MODE = 0 _ Autocorrelation 
MODE = 1 _— Cross-correlation 
MODE =2_~ Autocovariance 
MODE = 3 __ Cross-covariance 


FS Sampling frequency in Hz. 


IWIN Type of window used on correlation function. If IWIN = 1 a rectangular window is used; if 
IWIN = 2 a Hamming window is used. 


L Number of correlation points used in estimating the power spectrum. L must be less than or 
equal to M/2+1. 


NFFT FFT size (power of 2) used to compute spectral estimate from windowed correlation. NFFT 
must be greater than or equal to 2*L — 1. 


The user must supply the two routines GETX and GETY which provide values of x(n) and 0 to 
the main program. The calls to these routines are 
CALL GETX (X,NRD,SS) 
CALL GETY (Y,NRD,SS) 
where the parameters X, Y, NRD, and SS are 


X Array in which the samples of x(n) are stored 


¥ Array in which the samples of y(n) are stored 
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NRD Number of samples of x(n) (or y(n)) to be read into array X (or Y) 


SS __ Starting sample number of x(n) (or y(n)), i.e., samples x(SS) to x(SS + NRD — 1) are read 
into array X in the subroutine GETX. Note that SS is a floating point number to allow for max- 
imum flexibility. The first input sample is assumed to be x(1), i.e., SS = 1. 


The main program, CMPSE, and test subroutines GETX and GETY are given in the Appendix. 
The subroutine FFT is a complex FFT subroutine with calling sequence CALL FFT(X,M,INV), where 


M = Size of FFT (a power of 2) 
X = Complex array of length M. 
INV = 0 for direct FFT 


1 for inverse FFT 
2.3 Dimension Requirements 


The real array XA and the complex array X are dimensioned to size MAXM, where MAXM is the 
largest size FFT to be used in any run. The array XA is a data buffer and the array X is the working 
array for the FFT computation. The complex array Z is used to accumulate the transform of the corre- 
lation estimate. The real array XFR is used to generate a frequency scale corresponding to the DFT 
sample frequencies; the real array ILAG is used to generate a set of lag indices for printing. The arrays 
Z, XFR, and ILAG are dimensioned to (MAXM/2+1). 


2.4 Summary of User Requirements 


(1) Provide the subroutines GETX and GETY which specify x(n) and y(n) for the analysis to be 
performed. (For accessing data from a disk file, etc., the user must perform any necessary initiali- 
zation for reading the file.) 


(2) Provide the complex FFT subroutine FFT. 


(3) Specify the correlation parameters M, N, MODE, and FS, and the spectrum estimation parameters 
IWIN, L, and NFFT. : 


(4) Provide a graphical output to plot the correlation function and/or the power spectrum estimate. 
POWER SPECTRUM 
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Fig. 3. Plots of the log power spectrum and the autocovariance for Example #1, using a 
Hamming window. 


3. Test Problems 
To illustrate the use of this program a simple test problem is provided in which 
x(n) = cos(27n/10) 
and 
y(n) = sin(27n/10) 
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These sequences are sinusoids with frequency FS/10., where FS is the sampling frequency. 


Example #1 


For this example the program parameters are M = 128, N = 256, MODE = 2, FS = 10000, and the 
correlation parameters are IWIN = 2 (Hamming window), L = 32, and NFFT = 128. The numerical 
output of this example (i.e., the first M/2 + 1 points of the correlation function, and the M/2 + 1 
points of the log power spectrum) are given in Table 1. Figure 3 shows the autocovariance function 
and the log power spectrum for this example. Note the slight linear taper in the autocovariance func- 
tion estimate. This is due to the fact that only a finite number (256) of samples were used in the esti- 


mate. 
POWER SPECTRUM 


Sw EXAMPLE 2 
N AV: 


wile V 
|! 


23 


@ 
oS 
-57 
ie) 5000 
FREQUENCY IN HZ 
, AUTO COVARIANCE 
uw 
pe 
a 
> 


-{ 
0 64 
LAG IN SAMPLES 


Fig. 4 Plots of the log power spectrum and the autocovariance for Example #2, using a 
rectangular window. 


Example #2 


For this example M = 128, N = 256, MODE = 2, FS = 10000, IWIN = 1 (rectangular window), L 
= 32 and NFFT = 128. The numerical results are in Table 2, and Figure 4 shows the autocovariance 
function and log power spectrum. A comparison of Figs. 3 and 4 shows the well known fact that the 
Hamming window has a "main lobe" about twice as wide as the same size rectangular window, but much 


lower side lobes. 
POWER SPECTRUM 
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Fig. 5 Plots of the log power spectrum and the autocovariance for Example #3, using the 
Same parameters as Example #2 except NFFT = 512. 
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Example #3 


This case is identical to Example #2, except the FFT size, NFFT, was 512 instead of 128. Figure 5 
shows the autocovariance and log power spectrum and Table 3 gives the numerical results. Note that a 
much smoother looking plot is obtained. However, it is well to point out that this is nota more accu- 
rate representation of the true spectrum, but rather a more detailed representation of the estimate of the 


spectrum. . 
: POWER SPECTRUM 


— EXAMPLE 4 
“ee 2% f 


17 


0 5000 
FREQUENCY IN HZ 


CROSS COVARIANCE 


VALUE 


64 
LAG IN SAMPLES 


Fig. 6 Plots of the log power spectrum and the cross-covariance for Example #4 using a 
rectangular window. 


Example #4 


For this example M = 128, N = 256, MODE = 3, FS = 10000, IWIN = 1, L = 32, and NFFT = 
128. The numerical results are in Table 4, and Figure 6 shows the cross-covariance function and log 
power spectrum. Note that y(7) = x(n—2.5) so that R,,(m) should be roughly the same as the auto- 
covariance function of Example #2 but shifted to the right by 2.5 samples. This assertion is supported 
by a comparison of Figs. 4 and 6 with an allowance for the slightly different linear tapering and the half 
sample shift (interpolation). 

POWER SPECTRUM 
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Fig. 7 Plots of the log power spectrum and the autocovariance for Example #5. 
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Example #5 


CMPSE 


For this example the signal generators were programmed to provide samples of a white Gaussian 
noise (program not provided). Figure 7 shows plots of the power spectrum and autocovariance for the 


noise with the following parameters: 


M = 256, N = 4096, MODE = 2, 


IWIN = 2, L= 100, NFFT = 1024. 


A variation of about 5 dB is seen in the power spectrum for this theoretically white noise input. 
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Table 1 
M= 128 N= 256 MODE=2 SAMPLING FREQUENCY=10000.0000 
XMEAN= .69337E-06 YMEAN= .69337E- 
CORRELATION FUNCTION 
LAG CORR LAG CORR LAG CORR 


ee en UC 1 -403E 00 psa 
5 +. 492E 00 6 =. S358 oo 7 —.148E 0 
10-8822 0 14 3O7m Op. 12. .1458 0 
Ao = aise U0 oe Soe oe oT. Lae 8 
20,4632 00.21 ia? Pee GO 22° 2 1399k. 0 
49 ~.4538 00 .26 =.3632 00. 27 -.13GE 0 
30 .443E 00 31 oe od nt Ree > ees 2 7 ee 
35 -.434E 00 36 -,348E 00 37 -.130E 0 
40 .424E 00 41 -ae0n O00 42° ,127E: 0 
45. -.414E 00 -46 —-.332E 00 47 -.124E 0 
50 .404E 00 51 2282 00. 52 227218 2 
DS =.3908 00 56 —.316E 00: 57 =. 116E 
60 - . 3858-00 64 s008. 00... 62. .115E-0 
WINDOW TYPE=HAMG NO OF WINDOW VALUES= 
LOG POWER SPECTRUM 
FREQ DB FREQ DB 


~UD0 -21 4636 fe .tgo —52.g094 156. 
312.200 -21.1400..390.625 -=36.8481 468. 
625,000 =-29.4239., 2303-20 “14.9793 Ter. 
937.200 lJ. tage. Geo ADs Jae 1093. 

1250.000 -»,t3e to J320.125 —14 «p99 1406. 
1362. 500 -—38.4645. 3640.625 =25.3551 1748. 
1875.000 -—22.6829.1905.ig¢0. —29.A4415 2031. 
2i8/7.500 -30. 1628. 2265. 625 -32.3922 2343. 
2200.000 -31 .0/33,25/80.125 -34.7852 2656. 
2812.500 -31.8905 2890.625 -36.8850 2968. 
SA #05000 =32 5559. 3403.l2n ~—28.243/7 3281. 
ehod, 500 -—33.0708, 351 59.G25 =-40.7724 3593. 
3750.000 -33.4647 3828.125 -42.7547 3906. 
4062.500 -33.7648 4140.625 -44.9129 4218. 
4375.000 -33.9483 4453.125 -47.4943 4531. 
4687.500 -34.0617 4765.625 -50.7562 4843. 


5000.000 -34.0996 
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156E 
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Table 2 


M= 128 N= 256 MODE=2 SAMPLING FREQUENCY=10000.0000 
XMEAN= .69337E-06 YMEAN= .69337E-06 
CORRELATION FUNCTION 
LAG CORR LAG CORR LAG CORR LAG CORR LAG CORR 
0 .502E 00 1 .403E 00 2 .151E 00 4. 156R 00 224 + 4012 
5: =, 4925 00 6 -.395E 00 7. =. 1408 60 eo  tase 26:...9 393E 
10 ,482EF 00. 11. -.3878 00 ~ 12° °. 41458 O68 T3-- .150F 00:44 ~. 3865 
1§ «A758 00. 16 -—. 3798 20 7 =. ae 0 IB Ce 0 13 -3ISt 
20° {4638 000 24° SITTE' 00°22 .439E 00° 23-1448" 00 “24 370E 
25 =. 459% 00- 26 =.3638 00 27 =. 1368 00 -26 - 2812 00 29.3625 
$0. =, 44356 00. 31° -.3568 00 ~32° -4338 00 33 +~.33868 00 934 354E 
98 =. 4348 00 36 =. 3688 06-37. =. 1308 00 36: 21358-0606 59 346E 
40: .4248 00 41 .<3408 00 42" 4298 00. 43 -. 7328 00 44 338E 
45 ~.4146 00 -46 =-.3328 00 47 =-.1248 00 46° -.1298 060 49 330E 
50 -.4048 00. 5% 3288-06 So 4278-00: 53. =. 7268 00. 54 -. 3228 
55 -.395E 00 56 -.316E 00 57 -.118E 00 58 .123E 00 59 314E 
60 .385E 00 61 .308E 00 62 .115E 00 63 -.120E 00 64 -.307E 
WINDOW TYPE=RECT NO OF WINDOW VALUES= 32 FFT SIZE= 128 
LOG POWER SPECTRUM 
FREQ DB FREQ DB FREQ DB FREQ 
.000 1.2468 78.125 -29.4088 156.250 1.4597 234.375 -18 
312.500 2.2787 390.625 -13.7690 468.750 3.6083 546.875 -8 
625.000 6.1746 703 2425" +422933 °° 781.250 9.9020 859.375 5 
937.500 21.3157 1015.625 23.4804 1093.750 17.7880 1171.875 - 
1250.000 7.8799 1328.125 -2.6845 1406.250 4.0424 1484.375 -6 
1562.500 .7215 1640.625 -8.0628 1718.750 -1.1020 1796.875 -10 
1875.000 -3.0762 1953.125 -11.3458 2031.250 -4.1793 2109.375 -13 
2187.500 -5.5376 2265.625 -13.8561 2343.750 -6.2616 2421.875 -15 
2500.000 -7.2606 2578.125 -16.0150 2656.250 -7.7504 2734.375 -17 
2812.500 -8.5133 2890.625 -18.0259 2968.750 -8.8449 3046.875 -19 
3125.000 -9.4416 3203.125 -20.0211 3281.250 -9.6594 3359.375 -21 
3437.500 -10.1332 3515.625 -22.1209 3593.750 -10.2649 3671.875 -23 
3750.000 -10.6436 3828.125 -24.4750 3906.250 -10.7068 3984.375 -26 
4062.500 -11.0124 4140.625 -27.3203 4218.750 -11.0175 4296.875 -30 
4375.000 -11.2583 4453.125 -31.1973 4531.250 -11.2142 4609.375 -35 
4687.500 -11.4010 4765.625 -37.8602 4843.750 -11.3097 4921.875 -55 
5000.000 -11.4473 
Table 3 
M= 128 N= 256 MODE=2 SAMPLING FREQUENCY=10000.0000 
XMEAN = .69337E-06 YMEAN= .69337E-06 
CORRELATION FUNCTION 
LAG CORR LAG CORR LAG CORR ia CORR LAG CORR 
0 .502E 00 1 .403E 00 - Sie 00 3 -.156E 00 4 -.401E 
5 -.492E 00 6 -.395E 00 7 -.148E 00 8 .153E 00 9 393E 
iC 4878 UO 19°" 3878 00" 12° =<4458 00 13. =. 3508 00°74 -. 3865 
13 =~. 2798-00  T6 = 3798 00: “17 —, 1928 06. 48° ee 0 TS 378E 
20+? 463 YOO 27" STs OO ea | Se 00 a IE 24 SE 
o> A59E: OO 96 ~'. 963E 00 37 =. 136R 00 2a 2 4aik 00 39 362E 
$0°---. 4438 00-37 {S568 00° 32°> .1338: 6 Se BE 00 (34 ~.3548 
35°. eoak 00 > 36. SAGE 00 37>. 4308 oO Se SE OO: 9 346E 
40° +; 4048 00 - a4°*{ 580r 00 a2 *. 927 00 Se — 1338. 00: 44 = S388 
45° +.414r 00 46 -- 33928 00: 427 =. 4248: 00 “eB <1790F 00 ‘49 330E 
50 .404F 00 51 .324E 00° 52 .124F 00 53 -.126E 00 54 -.322E 
65°" 395k: 00° 56° —. S16E 00-57 =.418E 00 Se. 1238 00 59 314E 
60 > .385Ee 00-61 ° .30BE 00° 62°- .155E 00 63 =. 120E 00° 64 -.307E 
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Table 3 


(Continued) 


WINDOW TYPE=RECT NO OF WINDOW VALUES= 


LOG POWER. SPECTRUM 


FREQ 
700. 4 
78.125 -29. 
156.250 4 
234.375 -18. 
312.500. 2 
390.625 -13. 
468.750 3 
546.875 -8. 
625.000 6 
703.125 -4. 
781.250 9 
859.375. 5. 
937.500 21 
1015.625 23. 
1093.750 17 
$471 BIS 
1250.000 <5 7 
1329. 125° = 2) 
1406.250 4 
1484.375 -6. 
1562.500 
1640.625 -8. 
1748. 750 © 
1796.875 -10. 
1875.000 -3 
1953.125 -11. 
2031.250 -4 
2109.375 -13. 
2187.500 -5 
2265.625 -13. 
2343.750 -6 
2421.875 -15. 
2500.000 -7 
2578.125 -16. 
2656.250 -7 
2734.375 -17. 
2812.500 -8 
2890.625 -18. 
2968.750 -8 
3046.875 -19. 
3125.000 -9. 
3203.125 -20. 
3281.250 -9 
3359.375 -21 
3437.500 -10 
3515.625 -22. 
3593.750 -10 
3671.875 -23. 
3750.000 -10 
3828.125 -24. 
3906.250 -10 
3984.375 -26. 
4062.500 -11 
4140.625 -27. 
4218.750 -11 
4296.875 -30. 
4375.000 -11 
4453.125 -31. 
4531.250 -11 
4609.375 -35. 
4687.500 -11 
4765.625 -37. 
4843.750 -11 
4921.875 -55. 
5000.000 -11 


DB 


- 2468 


4091 


-4597 


6578 


-2787 


7696 


-6082 


5368 


-1145 


2924 


-9019 


4015 


- 3156 


4803 


. 1879 


3156 


287499 


6846 


-0424 


8825 


Sao 


0630 


1022 


Shes 


.0764 


3459 


.1794 


2568 


e379 


8558 


.2616 


4986 


- 2606 


0152 


<7303 


5443 


eb 


0261 


.8449 


5414 
4415 
0215 


-6594 
.6124 
.1334 


1191 


- 2642 


8953 


-6426 


4769 


«7036 


6516 


.0178 


3146 


.0190 


1636 


.2594 


19356 


mre 


8500 


-4021 


8505 


~3103 


6846 


-4473 


FREQ 
oe 
-656 
. 784 
. 906 
031 
136 
-281 
. 406 
os] 
-656 
7S) 
~ 906 
-031 
tS 
281 
- 406 
931 
.656 
-781 
- 906 
031 
<“T56 
-281 
. 406 
531 
- 656 
ag 
- 306 
031 
at es «, 
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a2 


FFT SIZE= 


. 469 


DB 


(5362 
-9062 
.7002 
. 8659 
soo23 
. 3086 
205 
.8471 
-7443 
58 
~9990 
~4202 
-6304 
Pto20 
91:38 
0190 
. 3834 
. 9980 
2099 
“6235 
haw 1 
2031 
. 1360 
. 2432 
-4598 
5 2905 
9787 
FT92 
-o196 
. 5614 
.8661 
.6007 
O92 3 
1583 
Re be ae Be 
:9756 
-4305 
~ooee 
.9483 
.0455 
12682 
. 3441 
.8918 
. 8941 
9421 
. 1094 
.1417 
Rae ge 
<052/ 
«1210 
9957 
» 259 
-4887 
- 2141 
.2846 
-2a16 
,O957 
. 5964 
«0559 
. 8858 
-8460 
. 8830 
.9108 
- 1289 
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Table 4 
M= 128 N= 256 MODE=3 SAMPLING FREQUENCY=10000.0000 
XMEAN= -69337E-06 YMEAN= -12023E- 
CORRELATION FUNCTION : 
LAG CORR LAG CORR LAG CORR 
0 =.286E-05 Ts (229358 -00 2 .472E 0 
5 -.427E-04 G-=—=.287E 0 7 -.462E 0 
70. +.588E+05 = 11 <2e,6 OD ¢ 2 .4538 0 
15. =.396E-04° 16 =.276E 00:5 17 —.444F 0 
20. =. 903E+05° 242° S270E 0052 22: .43588° 0 
25 -.374E-04 26 -.264E 00 27 -.425E 0 
30. +). 20E-—04 > 31 2O8E O0F Ss F2>= .4168: 0 
35..>+.3418-04 > 36 —.253E 00 © 37:+.4078. 0 
40 -.150E-04 41 -247E 00.42 .397E 0 
45 -.310E-04 46 -.241E 00 47 -.388E 0 
70. +. 168E+04 2 5151s 235E 007 * S22 .379E. 0 
35. =:. 291E-04 © 56 =.230E 00 = 57 +.370E 0 
60. —, 199E-04-¢ 61 242248 002° 62 2 .360E: 0 


WINDOW TYPE=RECT NO OF WINDOW VALUES= 
LOG POWER SPECTRUM 
FREQ DB FREQ DB 


000: ¢=959:759 78.125 = =1S696: 2456. 


al 2h 
625 
ge 
1250 
1562. 
1875 
2aB 7. 
2500. 
261 2, 
31255 
3437. 
3750 
4062. 
4375 
4687. 
5000. 


S00 -6.4814 390.625 -2.4477 468. 
.000 ~aQA SG; £70351 25° - =1310416 398 1:. 
200: 59.6. 7410731015. 625-4 7511391093. 
.000 2.67 13. 1328:.125° =19;.061 6: 1406). 
500 -—»6206 1640.625°=17.6371 1718. 
000. 2.-+3.8123. 49535125 =15°.97867 0: 2031< 
900 =-5;8886 2265. 625 —4 552223442343, 
0003 2=+7.3826: 2578.12 =15.03387) 2656. 
500° +8; 5420 2890. 625: -14,90642° 2968; 
0002.2 +92.3652: 3203.125° -1459200: 82615 
500: =1 0: 0635. 3515; 625: =1 4, 8582-32593. 
-000 -10.5840 3828.125 -14.7564 3906. 
500 -10.9704 4140.625 -14.6016 4218. 
-000 -11.2367 4453.125 -14.3844 4531. 
500 -11.3933 4765.625 -14.0969 4843. 


000 -11.4447 


2.2-10 


01 
LAG CORR 
0 3 .470E 
0 8 -.461E 
0 135° ..454£ 
0 18 -.442E 
0 23°° .433E 
0 22 - <423E 
0 33 .414E 
0 38 -.405E 
0. 43°-..396E 
0 48 -.386E 
0 525° s977£E 
O 658: 23665 
OD 63°“ .sek 
32 FFT SIZE= 
FREQ DB 
250 Tact SA 2 
750 Jf 05S 
250 F.BA2 2 
750 15.8456 
250 2.0385 
750; -+3.4963 
Bore). OoD2 
750-4 0, 5349 
250; #4 257978 
150: =1 4,617.78 
250: =16.0888 
75 +17. 2664 
250: 38.4832 
41502 =A 83:8550 
250; =) 9520999 
150533 9.62216 
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234. 
546. 
B5o. 
Ee 3 
1484. 
1796. 
ZI S, 
2421. 
2734. 
3046. 
33595 
S67 16 
3984. 
4296. 
4609. 
4921. 
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A Coherence and Cross Spectral Estimation Program* 


G. Clifford Carter and James F. Ferrie 


Naval Underwater Systems Center 
New London, CT 06320 


1. Purpose 


This program can be used to estimate various second order statistics between two real sequences. In 
particular the auto and cross power spectral density functions together with coherence and generalized 
cross correlation functions can be estimated. 


2. Method 


The method uses the overlapped fast Fourier transform (FFT) technique discussed by Carter, Knapp 
and Nuttall [1]. The method discussed is sometimes referred to as the direct method (as opposed to 
the indirect correlation methods) and has been discussed in part by Welch [2], Bingham, Godfrey and 
Tukey [3], Benignus [4], Nuttall [5], Williams [6], and more fully by Carter, Knapp and Nuttall [1]. 


The method begins with two (one from each process) digital waveforms (or with analog waveforms that 
have been lowpass filtered and digitized). Briefly, there are four steps in the estimation procedure: 
First, each time series is segmented into N segments, each having P data points. Second, each segment 
is multiplied by a smooth weighting function. Third, the z transform of the weighted P-point sequence 
is evaluated on the unit circle in the z-plane. Finally, the discrete Fourier coefficients obtained in the 
third step are used to estimate the elements of the power spectral density matrix by averaging "raw" 
power spectral estimates over all the N segments. A summary flow chart is given in Figure 1. 


More expiicitly, two random processes that are jointly stationary over N data segments are processed as 
follows (Carter and Knapp [7]): 


1. Each of the two time series is segmented into N segments of P points. The segments may either be 
disjoint or overlapped. Then one segment of P data points with the same (or aligned) time origin is 
selected from each of two time records. Even if each of the N data segments is large, P should be 
selected to ensure that the sampling frequency divided by P will afford sufficiently fine spectral resolu- 
tion. . 


2. Each of the two P point segments is multiplied by a smooth weighting function. Here smooth means 
that as many higher order derivatives be continuous as reasonable over the full interval of data points. 
The smoother the weighting function, the more rapidly the side lobes of its Fourier transform, or win- 
dow function, will decay, and thus the less leakage there will be extraneous power, which corrupts spec- 
tral measurements. Hence, smooth weighting functions result in better spectral estimates. Smooth 
weighting functions however, result in poorer frequency resolution when P is held fixed. If better reso- 
lution is desired, more data points per segment will be required. From a stability point of view, 
increasing P decreases the available number of independent data segments when the data duration is 
finite. 


The application of some smooth weighting function, has been shown to be necessary to reduce errors 
due to side lobe leakage. (Carter and Knapp [7]). However, weighting functions have the apparent 
disadvantage of wasting the available data. This apparent wastage can be overcome through the added 


* This program, originally written and documented by C. R. Arnold, G. C. Carter, and J. F. Ferrie, has been rewritten and 
tested by J. C. Sikorski, G. C. Carter, R. G. Williams, and J. F. Ferrie. U.S. Government Work not protected by USS. 
copyright laws. 
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START NNN. - NUMBER OF DATA 


POINTS (P) PER SEGMENT 
ISR - SAMPLING RATE (Hz) 


READ INPUT }— ————--—- NDSJP - NUMBER (N) OF DISJOINT 
PARAMETERS (NON OVERLAPPED) SEGMENTS 


SFX, SFY - DATA SCALE FACTORS 
USUALLY SET EQUAL TO UNITY 


READ A SEGMENT 
OF 2-CHANNEL DATA FROM 
MAGNETIC TAPES 


MULTIPLY DATA SEGMENT 
BY COSINE BELL 


COMPUTE NNN 
POINT FFT OR 
PAM-C2ZT 


BACKSPACE 
DATA TAPES 
ACCORDING TO 
PERCENTAGE OVERLAP 
DESIRED 


ESTIMATE SPECTRAL 
DENSITY MATRIX 


UPDATE RUNNING 
SUM OF ESTIMATE 
MORE 
DATA 


NO 


COMPUTE MAGNITUDE-SQUARED 
COHERENCE ESTIMATES 


PRINT INPUT PARAMETERS AND RESULTS 
STOP 


Fig. 1 Summary flow chart for coherence and spectral estimation program. 


cost of overlapped processing. Nuttall [5] has shown that the same equivalent degrees of freedom can 
be obtained from a fixed amount of data via overlapped processing as with indirect correlation process- 
ing for both auto and cross spectral density estimation. 


3. The transform of the weighted P-point sequence is evaluated on the unit circle in the z plane. The 
two sided z-transform of an infinite sequence is defined by Oppenheim and Schafer [8] as 


L 2a, (1) 


X, (z) = x x,(p)z?, on 


p=-© 
where z equals any complex variable. 
Similarly, Y,(z) is defined as the z-transform of y,(p). When x,(p) and y,(p) are finite in duration, 


the infinite series Eq. (1) becomes finite. Evaluation of the z-transform at P equally spaced points 
around the circle yields the discrete Fourier transform or DFT: 


X,(k) = ¥ x, (p)en204/? (2) 
p=0 


Similarly, Y,(k) is the DFT of the n'” weighted data segment y,(p), p = 0, 1,...,.P — 1. The DFT can 
rapidly be evaluated by the FFT. Having computed the DFT, Eq. (2), by an FFT, we are ready to 
proceed with the fourth step in the spectral estimation algorithm. | 


4. The spectral estimates are 
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= N 
G(k) =¢ ¥ 1X0, (3a) 
n=1 ; 
N 
6 =c ¥ YAOI, (3b) 
n=] 
. N - 
Gyy(k) = cg Y Xn(k) Y, (kK) , (3c) 
n=1 
with the constant gain 
C, = a, mg (3d) 


f, = sampling frequency and * indicates complex conjugate. The estimate of magnitude squared coher- 
ence (MSC) 


2 | Ge (k) |? 
CoA) me sre ert. 4 
ok) Gi. (k) Gy, (Kk) me 


Without some understanding of the statistics of the estimator in Eq. (4) it would be of little practical 
value. Under certain assumptions the statistical characteristics of C can be evaluated. These charac- 
teristics are based on the derivation by Goodman [9] of an analytical expression for the probability den- 
sity function of the magnitude coherence estimate and the subsequent extensions to magnitude squared 
coherence by Carter, Knapp and Nuttall [1]. These results are based on two zero-mean stochastic 
processes that were jointly stationary, Gaussian, and had been segmented into N independent segments. ! 
Each segment was assumed large enough to ensure adequate spectral resolution. Further, each segment 
was assumed perfectly weighted (windowed), in the sense that the Fourier coefficient at some kK” fre- 
quency was to have "leaked" no power from other bins. The statistics do not hold at the zero-th or 
folding frequencies (Hannan [10]). Extensions to Goodman’s work are given by Haubrich [11], Net- 
theim [12], Wahba [13], Carter, Knapp and Nuttall [1], and Halvorsen and Bendat [14]. 


3. Program Description 


The main program estimates the auto and cross spectral density functions. The programs listed are 
intended to be general FORTRAN IV programs; they have been compiled and executed on the PDP- 
11/70, System Engineering Laboratory (SEL) 32/55, Univac 1108, Control Data Corporation (CDC) 
6600 and International Business Machine (IBM) 360. The spectral estimation programs have been used 
for research projects by: Williams [6], Brady [15], Carter, Knapp and Nuttall [1], Carter, Nuttall and 
Cable [16], Santopietro [17], Carter and Knapp [7] and Carter [18]. These research projects were con- 
ducted entirely on the Univac 1108 and a significant program rewrite was undertaken to make the pro- 
grams more transferable from one computer system to another. The programs as a complete data pro- 
cessing system consists of input, computations and display. We have concentrated our rewrite efforts 
on the computations; both the input and display programs are expected to contain peculiarities of the 
particular computer being used. The input and display subroutines are modular so that only a 
minimum rewrite is required to transfer the program to another installation. The function of the input 
subroutine LOAD is to load the XX and YY arrays each with a real data segment of P (equivalent to 
the FORTRAN variable NNN) data points. If the data were stored on logical magnetic tape number 6 
in binary format the call to LOAD could be replaced by the FORTRAN statement 


READ 6, XX(I), YY(1), I=1, NNN. 


The subroutine LOAD listed in the Appendix is used to generate synthetic data for a suitable test case. 
1. Despite the fact that it is mathematically tractable to obtain analytic expressions only when the segments are independent, we 
would in practice use some overlapped processing to regain the apparent data wastage incurred by the necessity of data 


weighting. Carter, Knapp and Nuttall [1] report the results of an empirical study that demonstrates how bias and variance 
decrease as a function of increased data segment overlap. Fifty percent overlap is recommended with cosine weighting. 
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The main program also calls (in addition to LOAD): COPLT, HICMP, FFT842, PRCES, PRPRT, 
LREMV and ZERO. The subroutine ZERO is used to store zeroes in an array. The subroutines 
PRPRT and COPLT are display subroutines. The subroutine LREMV estimates (and optionally 
removes) the linear trend and mean value for the input time waveforms. These computations are per- 
formed for every time segment and are printed out by the main program as in aid to detecting nonsta- 
tionarities or digitizing errors. 


The subroutine FFT842 computes the FFT (see, for example, Cooley and Tukey [19]) coded and 
listed by Bergland and Dolan [20]. FFT842 can be replaced with a different FFT program such as the 
mixed radix FFT of Singleton [21]. Note that the subroutine PRCES and the main program presume 
that the FFT output array is subscripted from 1 to NPFFT and not from 0 to (NPFFT—1), where 
NPFFT is the number of point FFT to be performed. 


The subroutine PRCES implements the generalized cross correlation functions discussed in Knapp and 
Carter [22]. The subroutine PRCES calls on the subroutines FFT842 and COPLT. Given the FOR- 
TRAN variable NNN, the subroutine HICMP finds the highly composite number closest to (but greater 
than or equal to) NNN. The output of HICMP is NEWNN. For FFT842, NEWNN is 2 to an integer 
power. (For Singleton’s FFT see Carter [18].) For some applications, the program user will want 
NEWNN to be twice as. large as NNN; this is because the main program fills the data arrays with zero 
from NNN+1 to NEWNN. Such zero filling is (theoretically) required to inhibit the effect of circular 
convolution; in practice, though, (with stochastic data) zero filling often does not warrant the added 
computational cost. If it is desired, zero filling can simply be achieved by adding one line to HICMP: 
"NEWNN = 2*NEWNN". In addition to calling several critical subroutines, the main program per- 
forms computations necessary to estimate the spectral characteristics of the two waveforms under inves- 
tigation. 


When the two input waveforms are complex, one FFT of each waveform segment is required as 
specified in earlier. In most (though not all) practical data collection facilities, the input waveforms are 
real (not complex). When x(t) and y(t) are real, one FFT of the complex waveform x(t) + j y(t) 
can be computed and quickly be manipulated to form the FFT of x(t) and the FFT of y(t). (See pp. 
333-334 of Oppenheim and Schafer [8]; see also pp. 271-293 of Rabiner and Rader [23]). These obser- 
vations, combined with Eq. (3) give rise to the FORTRAN statements used to estimate the spectral 
characteristics of x(t) and y(t). The application of this theory reduces the computation time for two 
real waveforms by a factor of two. 


The final comment necessary before presenting the computer listings is to describe the input FOR- 
TRAN variables. NNN is the number of data points per segment. ISR is the integer sampling rate 
(Hz). NDSJP is the number of disjoint segments in the total time waveform. SFX and SFY are scale 
_ factors used to adjust the level of the input waveform to correct for frequency independent attenuations 
in the data collection and digitizing process. (When no correction is desired, the user sets 
SFX=SFY=1.0. When the user desires the spectral estimates to appear twice as large in power, he 
sets SFX=SFY=2.0.) With these five sample inputs, the input time data are processed. 


4. Test Problem 


The test problem was run with NNN = 32, ISR = 256, NDSJP = 8, SFX = 1.0 and SFY = 1.0. The 
LOAD subroutine used to generate the synthetic data is the one given in the Appendix. Selected out- 
put is given in the following tables. 
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Spectrum Analysis Chapter 2 
Table 1 
32 256 8.51.0 1.0 
Table 2 
NNN = 32 ISR = 256 NDSJP = 8 
SFX = 0.10000000E 01 SFY = 0.10000000E 01 
THE 8 DISJOINT PIECES COMPRISE 1.00 SECONDS OF DATA 
NUMBER OF POINT FFT = 32 
1 PRINTOUT OF FIRST 50 VALUES OF INPUT DATA 
1 . 92138798 3.00000000 
2 06441755 -0.12499915 
3 .80970714 ~ 1.00000000 
4 .54630972 3.00000000 
5 .00000000 3.00000000 
6 .25065174 3.00000000 
7 .00000000 1.00000000 
8 .00000000 1.00000000 
9 1.00000000 1.00000000 
10 1.00000000 1.00000000 
11 -~1.00000000 1.54018403 
12 -1.00000000 0.83475567 
13 -1.00000000 1.34758016 
14 -1.00000000 1.00000000 
15 0.66246481 0.71841439 
16 0.44273215 2.02324235 
17 0.60807347 1.00000000 
18 -1.00000000 2 .28788233 
19 0. 39938986 0.08707205 
20 0.78639817 -0.00627428 
21 -1.00000000 0.76997684 
22 0.84838115 3.00000000 
23 -0.57563173 3.00000000 
24 -0.00635505 1 .66902228 
25 -0.91894575 3.00000000 
26 1.00000000 0.16576968 
> 1.00000000 -~0.10451246 
28 0.69710672 1.00000000 
29 1.00000000 0.96631333 
30 -0 .63129364 3.00000000 
31 -0.14878848 1.00000000 
32 -~1.00000000 -0.12384148 
, 
1 DX= 0.63539E-01 DY= 0.14078E 01 SX=-0.69924E-02 SY=-0.20702E-01 
VX= 0.69209E 00 VY= 0.32296E 01 0 
2 DX=-0.41329E-01 DY= 0.12917E 01 SX=-0.22229E-02 SY=-0.16657E-01 
VX= 0:74174E 00-VY= 0.29968E 01 or 
3 DX=-0.21002E-01 DY= 0.14245E 01 SX=-0.54174E-02 SY=-0.23483E-01 
VX= 0.69332E 00 VY= 0.33341E 01 0 
4 DX= 0.41618E-01 DY= 0.14021E 01 SX=-0.96131E-02 SY=-0.67735E-02 
VX= 0.63633E 00 VY= 0.37315E 01 0 
5 DX= 0.10607E 00 DY= 0.14050E 01 SX= 0.12152E-01 SY= 0.27676E-01 
VX= 0.68438E 00 VY= 0.35066E 01 0 
6 DX=-0.55038E-01 DY= 0.12663E 01 SX= 0.47732E-02 SY=-0.69094E-02 
VX= 0.76118E 00 VY= 0.26802E 01 0 
7 DX=-0.12450E 00 DY= 0.13886E 01 SX=-0.82681E-02 SY=-0.18342E-01 
VX= 0.67879E 00 VY= 0.35007E 01 0 
8 DX= 0.41899E-01 DY= 0.13454E 01 SX=-0.15682E-01 SY=-0.15694E-01 
VX= 0.76327E 00 VY= 0.32313E 01 0 


AVERAGE VARIANCES ARE, VX=0.706388E 00 VY=0.327634E 01 
INTEGRATED VARIANCES ARE, VX=0.751587E 00 VY=0.455639E 01 


PLOT OF AUTO SPECTRUM GXX 
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FMIN 
INDEX 
is 
Fss08 
3 16. 
4 24 
$32 
6 40 
7 48 
8 56 
9 64 
10° 73 
se ee 
12 88 
13 96 
14 104 
415-442 
16 120 
17 128 
FMIN 
INDEX 
+8, 
24 
oe 
‘—* 
5 0 
6 0 
7S 
a © 
9 0 
10 0 
1454 
se ae 
13-5 
1429 
15 0 
16 6 
‘7 2 
FMIN 
INDEX 
, 
> ee 
3-6 
4.24 
5 "92 
6 40 
7 48 
8 56 
9 64 
9°92 
14-80 
12 88 
13 96 
14 104 
15 112 
16 120 
17 128 


Table 2 
(Continued) 
= -32.49 DB PEAK = -22.76 DB 
FREQ DB 
; -28.69 * 
.000 -22.76 «xxx 
.000 -24.14 *x*** 
.000 -23.85 *«x** 
.000 -25.94 «xx 
.000 -23.66 **** 
.000 -23.82 «xxx 
.000 -30.54 * 
.000 -28.08 «x 
.000 -22.88 **x* 
.000 -28.47 xx 
.000 -30.59 * 
.000 -32.49 x 
.000 -24.22 «xxx 
.000 -23.31 *«*** 
.000 -30.09 « 
.000 -28.35 «x 


PLOT OF AUTOCORRELATION RXX 


=-0.17620937E 00 PEAK = 0.10000000E 01 


TIME VALUE 
1. DODD KK KKH KKH HK K KKK HHH KKH HHH KKK KKH KKK HER KKKEKEKEK 
.00391 0.1584 KX HHHKKHHHKEX 
.00781 0.0703 *xxxHHKH 
01172 -0.0385 x#x«« 
.01563 -0.0151 xxxx«%* 
.01953 0.0123 *xxx*x** 
.02344 -0.1025 xx 
.02734 0.1921 KKH HK KHKHHKKHK 
.O031T25. -0,7762% 
03516 <=-6.1458 = 
.03906 0.0042 x«xxxxx 
.04297 0.0480 *xxxxxx% 
. 04688 0.0086 *«xx**** 
.05078 -0.1496 * 
.05469 0.0149 xxx xxx 
.05859 -0.0865 «xx 
-06250 -0.1143 ** 

PLOT OF AUTO SPECTRUM GYY 
«-=-27.55 DB PEAK =. --7.77-BDB 
FREQ DB 

—7 77 RRRHKHHES 
000 -13.28 *«*xxxx*x 
000 -20.87 «xx 
000 -22.16 «x 
000 -22.86 «x 
000 -21.48 «xx 
000 -22.34 «x 
000 -23.57 * 
000 -23.81 * 
000 -23.12 x** 
000 -26.30 «* 
000 -25.31 * 
000 -26.21 * 
000 -21.52 «** 
000 -22.17 «** 
000 -27.55 « 
000 -26.22 x* 


PLOT OF AUTOCORRELATION RYY 
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Table 2 
(Continued) 
FMIN = 0.18801017E 00 PEAK = 0.10000000E 01 
INDEX TIME VALUE 
1 0. Lt. GU00 KKK HHH HK HHH HH HHH KH KH KH HHH KH KKK HK HK KKH KKK K KKH 
a @580391 0.6667 HHH KKH KKK KKH KKK HHH HERE KEE 
S60 781 0.6123 KHHKHHEHHHHHREHHEREREREKHS 
2-0 07 172 0.5594 HHH KKKHHKKHKHKKKHKHRHE 
5 -§ 01563 0.5 2D5 KHHHKKHKHHKKKKHHKKHKK ; 
6 @. 01953 0.5461 KKK KH HH KK KKK HKKHHHK 
7: 0,823448 0.4426 HHH KKHKHKHKHEX 
8: -@. 02734 0.4713 KHHKKKHHHKKHKKKHEK 
oS 0.031725 0.3698 KxHKKHKHHKE 
+0 - 7.03516 0.3382 *xxxxK HX 
1t- .0.03906 0.3196 *xxxxx%% 
12 0.04297 0.2811 *x*xx 
13 6©§6©60.. 046886 0.2830 *xx*xx 
14 0.05078 0.2195 x 
15 0.05469 0.2356 «x 
16 0. 0565S 0.2086 x 
17. =~0.96250 0.1880 x 
1 DUMP OF CONTINUOUS PHASE VALUES IN DEGREES 
FREQUENCY 
2 ae Go; 
8.000 46.59 
16.000 48.43 
24.000 139.80 
32.000 140.38 
40.000 244.10 
48.000 . 251.48 
56,080 29 tet 
64.000 402.70 
72.000 413,14 
80.000 426.34 
88.000 486.45 
96.000 495.55 
104.000 670.33 
112.000 616.86 
120.000 637.78 
128.000 720.00 
1 DUMP OF THE MAGNITUDE SQUARED COHERENCE 
FREQUENCY 
0. 0.02 
8.000 277 
16. 000 0.46 
24.000 0.66 
32.000 0.42 
40.000 0.57 
48.000 0.66 
56.000 0.33 
64.000 (G5 
72.000 0.60 
80.000 0.49 
88.000 0.54 
96.000 0.44 
104.000 0.66 
112.000 O. 73 
120.000 0.08 
128.000 U3. +2 
1 
PLOT OF CROSS SPECTRUM GXY 
FMIN = -34.40 DB PEAK = -21.84 DB 
INDEX FREQ DB 
1 0. -27.20 «x 
2 8.000 -21.84 xxxxxx 
3 16.000 =24.21 #%*«+ 
4 24.000 -23.92 xxx 
5 32.000 -26.28 xxxx 
6 40.000 -24.05 xxxx«x 
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Table 2 
(Continued) 


7 48.000 -23.99 xxxxx 
8 56.000 -29.45 xx 
9 64.000 -32.38 * 
10 72.000 -24.10 xxxxx 
11 80.000 -28.93 xx 
tz 88.000 -29. 30 * 
+3 -96..000 -=34.. 2 = 
14 104.000 -23.76 xxxxx 
15 112.000 -23.44 xxxxx 
16 120.000 -34.40 x 
17 128.000 -31.74 x 


PLOT OF SCOT FUNCTION 


FMIN =-0.27144101E 00 PEAK = 0.10000000E 01 


INDEX TIME VALUE 
-15 -0.05859 -0.0367 *«*%xxx%x 
-14 -0.05469 0.0685 KHHKKKH HHH 
-13 -0.05078 -0.0173 ****xx%* 
-12 -0.04688 -0.0309 *x*xxxxx«xx 
-11 -0.04297 -0.0413 «**xxx«* 
-10 -0.03906 -0.0666 **xxxxx 
-9 -0.03516 0.0150 *xx*KxHHK 
-8 -0.03125 -0.1832 xxx 
-7 -0.02734 -0.0906 ***xxx 
-6 -0.02344 -0.0721 *xxxx«x 
-5 -0.01953 -0.1263 ***«x 
-4 -0.01563 1. OD00 KX KKK KKH KKK KKH KKH KKK HK HK KKK HK KH KKK KKK KKK KKK 
-3 -0.01172 0.0884 xxx KKK KKKKH 
-2 -0.00781 -0.0508 *xxx*xx 
-1 -0.00391 D..O785 sae ek ees HEX 
0 
2 
3 
4 
= 
6 
7 
8 
9 


0. -0.0964 xxxxxx 
0.00391 -0.0254 *xxxx*** 
0.00781 0.0076 **x**x%%* 
0.07172 0.1127 KKK KKK KHKKHEE 
0.01563 --0:2714 * 
0.01953 0.0138 *xKKKKHHH 
0.02344 0.0477 HHH KHKHH HHH 
002734 0.0947 KKK HK KKKHKHK 
0.03125 -0.0031 ***xxxx*x** 
0.03516 -0.2503 x 
10 0.03906 0.0520 *KxX KH KKHKKK 
14° 0.04297 =0 2190 * 
12 0.04688 -0.0850 *«xxxxx 
+3 0.050738 0.1026 KX XK KK HKHKKHEHE 
14 0.05469 0.0808 xx KKH HK HKHKK 
15 0.05859 0.0955 KKK KK HKHKHKKHKK 
16. 0.96258 0.0334 4H HKKK HK 
; 
PLOT OF PHAT FUNCTION 
FMIN =-0.20107339E 00 PEAK: =-.0:.. TOOQOQDD0UGE. 07 
INDEX TIME VALUE 


-15 -0.05859 -0.1132 «xx 
-14 -0.05469 0.0576 Kx HHHHH 
-13 -0.05078 0.0087 *«xx*x*x** 
-12 -0.04688 0.1063 KKK KHHHHK 
-11 -0.04297 -0.0956 xxx 
-10 -0.03906 0.0106 *x*x*x*** 
-9 -0.03516 0.0342 *xx*xx*x 
-8 -0.03125 -0.0108 *xx*x«x«x 
-7 -0.02734 -0.1228 xx 
-6 -0.02344 0.0016 *«xx*x«xx* 
-5 -0.01953 -0.0710 *xxxx 


-4 -0.01563 1. ODD D KX KKH HH KKHK KKH KKH HHH HK KKK HHH KK HH KKK KEKE KKH KKH KK 


Zoo 
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a ee 
on = 9, 
fe 
: ee 
s oes * 
eaten 5 
See * Be 
4 4%. 
~ te? = 
6 0 
ee | 
ef 
ee 
5S Se 
ci: ees | 
12 
Le Bees” 
14 0. 
2 Es * 
16-0, 
1 

FMIN 

INDEX 
“12 0 
-14 -0 
= 35 <9 
=-Tt2 =O 
mt I i 
=~ =< 
sk ts, 
-8 =9 
et 
“6-9 
a SH 
-4 -0 
a ae 
-2 <9 
=i. +6 

0 

1 

2 

3 

= 

5 

3 

7 

8 

9 


ers 
ak 
OO: SO So o.oo a © 


FMIN 
INDEX 
=45 9 
=14 <0 
+s 0 


OOS ©: O'S 


Table 2 
(Continued) 

OT172 0.0834 xxx HHHHKH 

00781 0.0005 xxxxxxx 

00391 0.1309 KH HKKHKKKHEK 

0.0364 xxxxx xx 

00391 -0.0259 xxx 

00781 0.0367 xxx KHHHX 

01172 0.1267 HHH HHHHHH HS 

01563 -0.1574 « 

01953 0.0259 xxx HHH 
-02344 -0.0032 xxxxxx«x 
.02734 0.1261 KHHHKHKHKHHS 
-03125 -0.0266 *xxx«x 
-03516 <-0.2017 «* 

-03906 -0.0047 xxxxxxx 

-04297 -0.1278 xx 
-04688 -0.0534 xxxxx 
.05078 0.0720 xxx HHHHHH 
05469 0.0483 *xxxxx Hx 
.05859 0.1496 KX KH HKKHKHHHKK 

06250 0.0708 xx xxx HH Hx 


PLOT OF CROSS CORRELATION 


=-0.34397384E 00 PEAK = 0 .99999999E 


TIME VALUE 

.05859 0.0363 KHHKHKHHHHHH 
.05469 0.0346 KHHKHHHHHHHK 
-05078 -0.0797 xxx%xx« 

-04688 -0.0962 *xxxxxxx 

.04297 0.1154 HHH KHKHKHKHHKH KS 
-03906 -0.0943 xxxxxxxx 

.03516 0.0692 KHKHKKHHKHHKE 
-03125 -0.0789 xxxxxx«x 


02734  -0.0351 *x*xxH HHH 
.02344 0.0645 KHHXKKHHKHHKKHE 
.01953 0 .159D KXHKHKHKHKHKHHKHKS 
«04563 1. OD D00 KH HHH KKH HHH HHH HK HHH HK KKH KK HHE KKK HK HKK KH KKH HEN 
BItT2 0.2454 KKK KHKHHHKKHKKHHKHEE 
.00781 0.1985 HHH KKH HHH HHH HK HKHE 
.00391 0.0785 Kx KKK HHKHHEHES 
0.0137 HHH HHKRKHHE 
.00391 0.1357 HHH HK HKHKHKHKHES 
-00781 -0.0413 xxx xxx H xx 
sOIiTFZ 0.2720 KHKHKKHKKHHKHHKHKHKHKHEE 
«04563. =—0. 3037 = 
-01953 -0.1149 xxxx%x%x 
.02344 0.0263 KH HKHKHHHHH 
02734 0 .027TB KHHHHKKHHHHKK 
,O3125 0.0109 HHH HHHH HHH 


Va510 -0.3440 « 


-03906 -0.0676 xxxxx HHH 
-04297 -0.3070 x 
-04688 -0.2010 xxxx 
-05078 0.0053 xxx XH HHHKK 
.05469 0.0484 Kx KKKKKHHHHHKK 
-05859 -0.0226 xxxxx xxx 
-06250 -0.0096 xxxxxxx xxx 
PLOT OF IMPULSE RESPONSE 
=-0 .21953384E 00 PEAK = 0.10000000E 
TIME VALUE 


-05859 -0.0803 x«xxx¥x 
.05469 0.0746 *xXXXHHHHH 
-05078 0.0486 xxxx xxx 
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Table 2 
(Continued) 
-12 -0.04688 0.0800 **xxKHKHKHX 
-11 -0.04297 -0.0617 **xxx 
-10 -0.03906 -0.0118 ***xx«x 
-9 -0.03516 0.1547 KHKKKHKKKKKKS 
-8 -0.03125 -0.1106 «xxx 
-7 -0.02734 0.0197 xxx HH 
-6 -0.02344 0.0170 *x*KxxH 
-5 -0.01953 -0.0603 «xxx 
-4 -0.01563 1. DODD KKK KKK KH HHH HEHE KKK HHH KKK KEK HE HEHEHE KKK KKH HHH KEK KKK 
-3 -0.01172 0.1841 HHH KHKHHHKKKKKK 
-2 -0.00781 0.0015 *xxx*x** 
-1 -0.00391 0.1916 KHK KH KKHKKHKHEK 
By -0.0267 x*xx*** 
1 %.00391 0.0846 KxxKHKHKHK 
2 BETS) 0.1157 KHKKHHHHEHKEK 
S* 2 O2E1 Tse 0.0311 xxx HHHE 
4 0.01563 -0.1551 «x 
5. C.879s3 0.1101 HHH HHHHKHKK 
6 0.02344 0.0212 xx xxxx**x 
7. -B 02756 0.0761 xxx HHHX 
8 0.03125 -0.0276 **xxx** 
oS §.0g516..-<0..2795: » 
10. 9.033906 0.0860 *xx*HKXKKHHX 
11 0.04297 -0.1769 x 
12 0.04688 -0.0984 «xxx 
+3 - 0, 05078 0.0904 Kx KKH HHKKKK 
14 0.05469 0.0731 KX HHH HH 
15 0.05859 0.1506 Kx KHKHHHHKKKH 
16° 0.86250 0.0278 «xxx xxHKX 
1 ; 
PLOT OF H-T(I) FUNCTION 
FMIN =-0.41350540E 00 PEAK = 0.99999999E 00 
INDEX TIME VALUE 
-15 -0.05859 0.1616 KHK KH KKKHHKHKKHKHEK 
-14 -0.05469 0.1164 KH KKHKKHKKKKHHES 
-13 -0.05078 0.0293 KH HKHKKHKHKKH HK 
-12 -0.04688 -0.2013 *«*xxxx 
-11 -0.04297 0. 0462 KHHKKHKHHKKKKH 
-10 -0.03906 -0.1862 *x*xxxx 
-9 -0.03516 0.0473 KHKHKHKKHKHKHEE 
-8 -0.03125 -0.2488 xxxxx 
-7 -0.02734 -0.0879 xx *xxx%xxx 
-6 -0.02344 -0.0597 xx xxx %%%H%% 
-5 -0.01953 -0.1886 *«xx*x«xxx 
-4 -0.01563 1. DODD D0 KKH KKK KKH KK KKH K KEK KKH HEHEHE KKK KKK KKK KKH HEHEHE KKH K KH 
-3 -0.01172 -0.0182 xx xHxxHHHHKH 
-2 -0.00781 -0.0176 *xxxHKKH HHH 
-1 -0.00391 -0.0216 KKK KHKKHKHKKK 
3 eee * ee -0.2124 xx xx«x 
p Paar fe 5 8 oS 0.0345 KHKKKKHKKKKKKKEH 
2 0.00781 -0.1292 *xxxxxKxx 
a. 20.09 +72 0 .241D KKK HK KHHKHKHKHKHK KH 
4 -0.01563 -0.4135 + 
a 284953 0.0438 KKK KKH HH HKKHHK 
6 0.02344 0.1171 KH KKKHKHKHKKKHHK 
T- S B2T34 0.1649 KKK KKHKKHK KKK HH KKK 
SS: - 2 b3325 0.1535 KHHKKKHKHKKKHKKHHKS 
o 6.03516 -8.3226 4% 
10 0.03906 0.1061 KH HX KKKKKHKHHKHHES 
+14 0.04297 -0.2780 «xxx 
12 0.04688 -0.0597 *x xxx HH 
13.0 056768 0.0888 KKK KKKKHKHKHKHKHK 
14 0.05469 0.0750 KXHKKKKKHKKHKKHE 
15 0.05859 -0.0103 *** HHH K HHH 
16-93. 962590 0.0463 KKK KKKKKKKKKHK 
1 


PLOT OF ECKART FUNCTION 
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Table 2 
(Continued) 
FMIN =-0.63544157E 00 PEAK = 0.10000000E 01 
INDEX TIME VALUE 
-15 -0.05859 0. 5FB4G KKK KKH HHH HHH KKH HK HKKKHKH HH HHH HKHHHHES 
-14 -0.05469 0.1941 KK KKK HKKKHKHKKHKHKKKHKHKES 
-13 -0.05078 0. 2D2QDD KHKKHHKKHHKKH HK KKK KHKHKHEK 


-12 -0.04688 -0.5636 x* 
-11 -0.04297 -0.3422 *xxxxxxx 
-10 -0.03906 -0.6354 x 
-9 -0.03516 -0.4095 *«xxxxx 
-8 -0.03125 -0.3856 *«*xxxxx 
-7 -0.02734 -0.3801 *xxxxx«*x 


-6 -0.02344 0.2161 HHH KK HKHKKKKEKKKHKKKKHEKK 

-5 -0.01953 0.2143 KHKK KKK HKKHKKHKK KKK KKH 

-4 -0.01563 FT .DO DD HHH HHH HHH HHH KH KKK KKK KK KK KKK KK KKKKKKKKKKKH HHH 
-3 -0.01172 0 47 2DD KKK HK HK HK KKK KKK KKH K HK KKH HH HHH 

-2 -0.00781 0D. 45 TG KKK KKH HHH KKH HHH KKK HH KKK HHH HHHEK 


-1 -0.00391 —0 0076 KX KK HHH HHHKHHKHHEK 


¢ Wes | —-0.1843 KX HKKHKHHHKH 
1 0.00391 —0 0194 KX KKKKKHKHHKHHKS 
2° BOTS -0.3895 *«*x*xxx 
3 0.01172 —-0.0398 KKK KKKHKH KKH HHH 
4. 0.01563 .=-0. 3455 = 
5 Bess 0.0318 KXKKKKHK HK KHKHHKKHEE 
G6 0.02344 —-0. 0044 KHHKKHKKHHKHHHHHH 
7 — 22 T s4 0.1 D6 7 KKK HH K HHH KKK HK KH KK HHEE 
BS .- 9; 03525 0.1180 HHH HKKKK HH KKH HHH KHHEK 
9 0.03516 -0.3017 *x*HHHHH% 
10 0.03906 -0.0853 xx KKH HH HKHHHHH 
11 0.04297 -0.3988 «xxx 
12 0.04688 0.21 5D KKH KKKHHKHKKHKHHHEKKKHHHE 
13. 0.05078 -0.0369 xX HHHHHKHHHHHEEE 
14 0.05469 0. 25GEG KKK K KKH KKH HHH KKK K HHH HEHE 
15: > D4 65e55 0.2253 KKK KKKHHHHKHHHKKKHKHHKHS 
16°  6,06250 0. 3553 KH KKH KKK KKH KKK HK KH KKK HHH HHE 
1 PLOT OF THE TRANSFER FUNCTION 
t 
FMIN = -8.64 DB PEAK = 2.97 DB 
INDEX FREQ DB 
1 0 2.97 *x*** 
2 8.000 1.84 *%*%* 
3 16.000 -0.14 «xxx 
4 24.000 -0.14 «xxx 
5 - 32: 8000 §=-0.69: +** 
6 40.000 ---0.77 x" 
7 48.000 -0.33 «xx 
8 56.000 2.18 ***x** 
9 64.000 -8.60 « 
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(LHL/) LVWuOd 7666 
(T666‘ULUYdI) ALIUM 
SANNILNOO O€ 
(x9 ‘8°SLaAZ ‘XL ‘9I) LYWHOd €666 
(I)AA °(1I)XX ‘I (€666‘ULHdI) ALIYM 
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CHAPTER 3 


Fast Convolution 


L. R. Rabiner 


Introduction 


The only program in this chapter is an implementation of a high speed (FFT) convolution as pro- 
posed by Stockham [1], and Helms [2]. The program was provided by J. Allen and is a direct imple- 
mentation of the method of overlap-add. The program is designed to handle efficiently large amounts 
of data assumed available on a file or disk. 
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FASTFILT -- An FFT Based Filtering Program 


Jont B. Allen 


Acoustics Research Dept. 
Bell Laboratories 
Murray Hill, NJ 07974 


1. Purpose 


FASTFILT is intended to be a general purpose utility program for filtering large amounts of data 
from disk. It consists of a MAIN program called FASTFILT and a subroutine RFILT which may be 
used independently if desired. The input data sequence is assumed to exist in a file on disk (as binary 
data). The function of FASTFILT is to sequentially read in blocked segments of data, filter them (by a 
call to RFILT), and store the results on disk in a second file. The filter is always represented by its 
impulse response.! (A test program for RFILT called TESTFILT has also been provided.) 


2. Method 
The filter technique is the well known method of overlap-add [1,2]. 


3. Program Description 


When specifying the filter impulse response to FASTFILT the user may elect to either read a 
precomputed impulse response from disk or input an impulse response from the teletype terminal 
input. In the following, disk files will be given as names in quotations, i.e., "DISKFILEIN", "DKOUT". 
The teletype terminal name on input will always be represented as "TTYI" and the teletype name on 
output as "TTYO". 


When entering the filter impulse response data from "IT'TYI" the program recursively asks for delay 
and gain values. The delay requested IDEL is the index of the impulse response array sample 
F(IDEL). The gain value requested is the value of the impulse response at delay IDEL. For example 
if IDEL = 1024 and G = .8, then F(1024) = .8. Any delay values not specified will have zero gain. 
IDEL = 1 represents no delay, while a gain of one represents no gain change. When IDEL is set to 
zero, the loop is terminated and the impulse response F(I) is stored on disk in a file called "FILT- 
DATA", thereby making it available for subsequent use. The filter length must be a power of 2, but 
this represents no restriction of course, since one may append any number of zeros to the impulse 
response. 


The length of the filter, NP, is limited by the maximum FFT size NMAX. The length of the FFT 
used NPT2 must be twice as long as the impulse response, NP. Thus, NP < NMAX/2. The total 
array space required is two real arrays NMAX long and a real array NMAX/2 long. Thus the total 
Space required is 5*NMAX/2 reals. If one real word is stored in two integer locations, the total integer 
array space required is therefore 5*NMAX or 10*NP. These numbers assume that a real to complex 
FFT is used, such as Bergland’s FAST, FSST, or FFA, FFS (see section 1.2). On our Data General 
Eclipse, each user has 32K words of 16 bit core. In this case NMAX may assume a maximum of 4096 
and the maximum filter length NP is 2048 points. Under these conditions the total integer array 
requirement is 20,480 words (leaving 12,288 words for the program code.) 


1. If the user has a filter design package he wishes to use with this program he must compute the filter impulse response 
using his program and store it on disk. 
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It is an important fact that the amount of time to perform the filtering per sample of input data 
increases only as log,(NPT2), the logarithm base 2 of the FFT length. Since the disk overhead reads 
and writes usually decrease as NPT2 increases, the time required for filtering is nearly independent of the 
filter length. 


The first question asked by our program is the filter length. This question is asked in order to keep 
the size of files stored on disk as small as possible: if disk storage were not a consideration, a standard 
2048 long impulse response could be assumed with no significant loss in speed. 


3.1 Data I/O 


This program has not been intended for a batch environment. (The batch user must use his own 
I/O and RFILT.) We assume that the user may respond to questions through keyboard input. 


Besides TTY I/O, it is necessary to read and write data from the disk. DISK I/O is done in one 
Routine DKIO. We assume that the user has named disk files and that commands are available to open 
named disk file on a FORTRAN channel for reading and writing. 


The subroutine DKIO must be supplied by the user. The arguments which must be supplied are (in 
order as they appear in the call) 


NAME = An array containing the name of the disk file to be accessed (defined by call GNAME) 
ICN = Fortran channel number of named file 

[BO = Relative block number to be read or written 

IARRAY = Integer array for core data storage 

NBLK = _ Size of IARRAY, in disk blocks 

IER = _ Error Flag (not used here) 

IOPER = Operation Code 


The final argument determines the operation to be performed according to the following table 


IOPER 


eS open disk file 


close disk file 
read from disk file 
write onto disk file 


Alternatively the user may rewrite the four calls FOPEN, CLOSE, RBLK, and WBLK. 


A remaining complication is that of reading the file names from the TTY keyboard. The problem is 
that a file name may be any number of characters long and thus some form of free format 
alphanumeric READ on "TTYI" must be available. In order to handle this we assume that the user has 
a subroutine called GNAME. When GNAME is called, the user types in a file name of any (reason- 
able) number of characters, and this name is returned in an array NAME, formatted so that it may be 
used directly when the file is opened. For example 


DIMENSION NAME (10) 
CALL GNAME(NAME) 
CALL DKIO(NAME,0....,0) 


opens FORTRAN channel 0 on the disk with a disk file name defined by the user in response to the 
GNAME call. 
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3.2 Speed 


The following times are for the DATA GENERAL ECLIPSE computer. Our 1024 point real FFT 
time is 288 (milliseconds, FFA time). The time required to filter 25,600 data points with a 2048 point 
filter is 40 sec. Thus 


time _ 40sec _ ms 


point 25600 — ==pOnat 


These figures may be used to compensate for the speed of the FFT on the users machine by propor- 
tional adjustment of this time. 


3.3 Accuracy 

The accuracy will be close to the accuracy of the FFT (see Tables 4 and 5 of Section 1.0). 
3.4 RFILT 

Subroutine RFILT is called as follows 
CALL RFILT(R,F,S,NP) 


where S is a scratch array NP points long, F is the filter frequency response 2°*NP+2 real (NP..+ 1 
complex) points long. R(1) to R(NP) are the input (prior to the call) and output data points (after 
return from the call), while R(NP+1) to R(2*NP) are data points left from the previous call to RFILT. 
Prior to the first call to RFILT, R(NP+1) through R(2*NP) should be zeroed. After the initial call to 
RFILT, this portion of R should not be changed in any way. The array S may be used in the main pro- 
gram for any purpose since it is only used as a temporary storage array. For example it may be 
EQUIVALENCE’d to an integer array for the disk read and writes. The array F is assumed to be the 
output of FAST (or FFA) corresponding to the filter input impulse response. The filter impulse 
response must be zero in the range of time samples n between NP+1 through 2*NP, i.e., the second 
half of F, prior to calling FAST (FFA). The user should study the listings for further details. 


3.5 FAST, FSST 


These FFT subroutines are those given in this book in Section 1.2. They may be replaced by FFA 
and FFS for greater speed. 


3.6 Further Constant Definitions Are: 


ITTO TTYO output channel number 

ITTI TTYI input channel number 

IDSKO, IDSK1 — allowed disk channel numbers 

NBLKL length in words of minimum disk block size (power of 2 only). 
NMAX maximum FFT size allowed 

SCALE A positive number equal to the maximum magnitude of data to be 


stored or read from disk (This number may be D/A and A/D dependent). 
NP Filter length 


3.7 Required External Calls and Functions 


CALL GNAME CALL FAST CALL DKIO 
CALL FSST IIMACH 


4. Test Examples 


In the following examples, computer responses are in BOLD FACE while user responses are in 
ITALICS. Comments are in normal type. 


Example 1 


For the following dialogue we assume that the filter impulse response is on disk in a file named 
"FILTER". We wish to filter some speech data also on disk in file "SPEECHIN". The output, the 
filtered speech data, will go to disk file "DKOUT". 
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DO YOU WISH TO DO FILTERING (=0) 

OR CORRELATION (MATCHED FILTERING) (=1) ? 
0 

PROGRAM WILL DO FILTERING 

FILTER LENGTH IN 256 WORD DISK BLOCKS (1,2,4,0R8) = 
8 

FILTER WILL BE 2048 POINTS LONG 

INPUT FILTER FROM DISK(1) OR TTY(2) = 

1 

FILTER FILENAME = 

FILTER 

SYSTEM GAIN = (F10.1) 

f, 

INPUT DATA FILENAME = 

SPEECHIN 

OUTPUT DATA FILENAME = 

DKOUT 

# OF DATA BLOCKS TO BE FILTERED = (I5) 

00100 

STOP 


Example 2 


As a second example we input the filter impulse response from the keyboard. All other conditions 
remain the same. 


DO YOU WISH TO DO FILTERING (=0) 

OR CORRELATION (MATCHED FILTERING) (=1) ? 

0 

PROGRAM WILL DO FILTERING 

FILTER LENGTH IN 256 WORD DISK BLOCKS (1,2,4,0R8) = 
8 

FILTER WILL BE 2048 POINTS LONG 

INPUT FILTER FROM DISK(1) OR TTY(2) = 


Z 

DELAY = 0 TO TERMINATE INPUT MODE 

DELAY = (14) 

0002 delay by one sample 
GAIN = (F10.1) 

-l. change sign of data 
DELAY = 0 TO TERMINATE INPUT MODE 

DELAY = (14) 

0000 


IMPULSE RESPONSE IS ON DISK IN FILE "FILTDATA" 
INPUT DATA FILENAME = 

SPEECHIN 

OUTPUT DATA FILENAME = 

DKOUT 

# OF DATA BLOCKS TO BE FILTERED = (15) 

00100 

STOP 


The data will be delayed by one sample and inverted as a result of these commands. 


5. TESTFILT 


TESTFILT is a main program written to test subroutine RFILT. It also provides an excellent test of 
the FFT routines. The program requires no input and upon completion prints a pass/fail message on 
the output device. It filters an internally generated function with the impulse response (0.,—1.,0.). It 
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then compares the result to the original input function. Required external routines are FAST, FSST, 
RFILT, ILMACH, and RIMACH. TESTFILT should be run before attempting to run FASTFILT. 
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CHAPTER 4 


Linear Prediction Analysis of Speech Signals 


B. S. Atal 


Introduction 


Linear prediction is one of the most widely used techniques in speech analysis.. It provides a simple 
and effective method of representing speech signals in terms of a small number of slowly-varying 
parameters. The linear predictability of speech signals is based directly on a linear filter model of 
speech production. In this model, the filtering properties of the glottal flow, the vocal tract, and the 
radiation at the lips are represented by a single time-varying all-pole filter. The linear prediction tech- 
niques provide a simple method for estimating the parameters of the all-pole filter. 


Linear prediction parameters have been found useful in a variety of applications, such as speech 
coding, speech recognition, speech synthesis, and speaker verification. Although the technique of 
linear prediction is applicable to a much broader class of signals, the programs presented in this section 
have developed from applications to speech signals. For applications to other classes of signals, great 
care must be exercised in choosing the right program and in the appropriate selection of the variables of 
the programs. Even for speech signals, there is still a large number of unresolved issues: How many 
predictor parameters should be used? What is the ideal analysis interval? Where should the analysis 
interval be located with reference to the pitch period? What is an ideal anti-aliasing filter for analog- 
to-digital conversion of the speech signal prior to LPC analysis? The answers to such questions often 
depend on the particular application for which the LPC analysis is used. However, it is fair to say that 
we still lack a clear understanding of the constraints which a physical signal must satisfy for the LPC 
analysis to work satisfactorily on that signal. 


The programs presented in this section are of two types: 1) a set of programs (AUTO, COVAR, 
CLHARM, and COVLAT) related to different methods of LPC analysis, and 2) another program, 
LPTRAN, that transforms one LPC representation to another. 


Several alternate ways of doing LPC analysis have evolved over the past ten years or so, mainly to 
take care of the complications introduced by the nonstationary nature of speech signals. For a nonsta- 
tionary signal, it is important to perform a “short-time analysis" [1] using short segments of the signal 
during which it could be considered to be nearly stationary. The differences in the various LPC analysis 
methods lie in the manner in which they treat the signal outside the analysis interval in the process of 
determining predictor coefficients. 


The correlations between different samples of a signal play a key role in the LPC analysis. All of 
the LPC analysis methods require that such correlations be computed in the process of determining 
LPC parameters. In the program AUTO (based on the autocorrelation method of LPC analysis), these 
correlations are expressed in terms of a short-time autocorrelation function [1,2]. This method requires 
that the signal be set to zero outside the analysis interval and a suitable window, such as a Hamming 
window [2], be used to reduce the abrupt change in signal values occurring at the beginning and at the 
end of the analysis interval. The use of an autocorrelation function leads to a particularly simple and 
efficient algorithm for determining LPC parameters. The program COVAR (patterned after the covari- 
ance method of LPC analysis) in the same section computes the LPC parameters by minimizing the 
mean-squared prediction error over the analysis interval. This method avoids truncation of the signal, 
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but does require that an entire matrix of covariances be computed from the speech signal. In its origi- 
nal form [3], this method did not always produce predictor coefficients which could be used for a later 
synthesis of the signal without producing unstable outputs. However, the program COVAR can be 
modified to produce a set of reflection coefficients, k,,k), ...,ky, which can be transformed to a set 
of predictor coefficients corresponding to a stable all-pole filter [4,5]. This transformation can be per- 
formed, if desired, by the program LPTRAN (to be discussed later). The programs COVLAT and 
CLHARM described in Sec. 4.2 follow a somewhat different approach to linear prediction based on the 
lattice structure of the all-pole filter. The lattice formulation does not need a knowledge of the short- 
time autocorrelation function for determining predictor coefficients and therefore avoids windowing of 
the signal. Also, the resulting all-pole filter is always stable. Thus, the lattice method may be prefer- 
able for certain applications. 


As expected, the LPC analysis produces accurate results if a signal has been generated as the output 
of an all-pole filter excited either by a delta function or white noise. However, it is rare that a physical 
signal can be modelled exactly as stated above. The performance of LPC techniques suffers whenever 
any one of the assumptions underlying the linear prediction model is violated. For example, in the 
prsence of spectral zeros (which could be introduced by the vocal tract, by the glottal excitation, or 
even by the anti-aliasing filter used in the A/D conversion), the linear prediction techniques not only 
skip over the spectral zeros but often introduce errors in formant frequencies and their bandwidths. 
The periodic nature of voiced excitation also introduces similar errors [3]. These errors must be kept 
in mind while interpreting the results of LPC analysis. 


There are several equivalent ways of describing the characteristics of an all-pole filter. The program 
LPTRAN described in Sec. 4.3 allows one to go from one representation to another in an efficient 
fashion. Although all these representations are equivalent and are uniquely related to each other, some 
of them may be more suitable than others for certain applications [2,3]. For example, the area function 
representation has been found to be well-suited for speech synthesis-by-rule applications where a con- 
tinuous speech message is created by concatenating and then smoothing a number of prestored ele- 
ments. Similarly, the cepstral parameters have been found to be most effective for speaker recognition 
tasks. Another important transformation is the set of partial correlation coefficients (PARCOR) or 
reflection coefficients. Table 1 of Sec. 4.3 lists the different LPC representations which can be 
transformed by LPTRAN. It should be noted that the transformations between alternate LPC represen- 
tations are valid only for all-pole filters. 
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1. Purpose 


The purpose of these programs is to implement the autocorrelation (AUTO) and covariance 
(COVAR) methods of linear prediction analysis. 


2. COVARiance and AUTOcorrelation Methods 


Two cases will be treated here, commonly termed the covariance and autocorrelation methods. The 
Fortran subroutines which carry out the solution are called COVAR and AUTO. For derivations, 
detailed discussion, motivation, and references the reader is referred to [1]--only a summary of the 
algorithms is presented here. 


The general problem can be stated as follows. Given a data sequence {x (0),x(1),...,.x(N—1)} and 
an integer M, find the coefficients {a 1,@>,...,a,;} which minimize the summation 


a= [Ix (n) + 5 a,x (n—k)]? . (1) 
n=N - 
This minimization is carried out by solving the equations 
y ac, =— co, tor k=—L2..M, (2) 
where 
Ci. = y x(n—i)x(n—-k) . (3) 
n=No 


In the covariance method the limits 1) and n, are defined so that no data points {x(n)} outside of 
the range 0 < n < Nare needed for the evaluation of Eq. (3): 


nyo= M, nj=N-1. (4) 


In this case, the coefficient set {c,,} forms a symmetric, positive semi-definite matrix. The matrix will 
be singular if the input data sequence {x(n)} satisfies a linear homogeneous difference equation of 
order M or less. 


In the autocorrelation method, the data sequence is treated as though it were zero outside of the 
interval from n = 0 through n = N — 1: 


x(n) =0 for n»<0O andfor n>N-—1 (AUTO). (5) 


In this case no and n, are defined as —c and +00 respectively. The coefficient set {c,,} forms the ele- 
ments of a symmetric positive definite Toeplitz matrix, and the coefficients can be expressed in terms 
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of an autocorrelation sequence as 
c, = r({i-k]) (AUTO) (6) 


where 


(i= SS aia eu hen Cet). (7) 
n=0 


Only values of r(k) for k =0,1,...,M are needed for the solution. 

The two methods give identical results when the data sequence is truncated so that x(n) =0 for 
n < Mand for n > N —1-—M. The two methods give similar results when N >> M. 
3. The Algorithms 


The subroutine COVAR implements a form of Cholesky decomposition, and the subroutine AUTO 
implements a form of Robinson’s recursion. As many of the steps are common to both, a common 
notation is used in this discussion. 


3.1 Initialization (step zero) 


For COVAR Egs. (3) and (4) are applied directly to obtain the covariance coefficients Cog, Cio, and 
C1}. For AUTO all of the correlation coefficients are obtained by directly applying Eq. (7), though only 
r(0) and r(1) are needed for initialization: 


Coo = Cy, = (0) and Cio = r (1) (AUTO) . 


The following parameters are then defined: 


ao = bo) = 1. 

ay = Coo, Bo = tii; 
ky a C11, 

aig. = 4, ay = 3G. 
a, = a — kf Bo. 


3.2 Recursion (step m for m=1,2,...,.M@—1) 


For COVAR, ©¢,,11.9 is directly evaluated from Eq. (3) and Eq. (4). The coefficients c,,4,, for 
k = 1,2,...,m+1 are found from 


Cntik = Omk—-1 + X(M—m—1)x(M—k) — x(N-—m—1)x(N—k) 
For AUTO, the correlation sequence {r(k)} is was evaluated in the initialization step. 


For COVAR, one evaluates then 


n+l 
Ymn = ; 2. bi Se ee (8a) 
Bn jal 
bnj ——s Y mi bi; for Fi — ae ’ (8b) 
j=j-1 
iain! =| , (8c) 
m+l 
Bin a 2 Cm+1,j binj . (8d) 
j=l 
While these equations are correct for the AUTO case, they are not necessary since 
by Ona psofor 2 eils2iym + doe (AUTO). (9a) 
and 
Bin =m - (9b) 


Finally, defining 
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Kosice = = : RES Pe (10) 
m i=0 


QAm+1,0 — , 
Qm+1,i = ami - Km+1 Dini for as Oe ree. ’ 


Gin +t tt  Retites 
and 
aga Oy — Keen 

step m is completed. 

3.3 Termination (end of step M—1) 

At the end of step M — 1, the results 
a, = ay, for k =1,2,....M 
and 
a=ay (11) 


are obtained. In addition, the intermediate parameter set {k,,k,...ky} is supplied as an output. These 
parameters are often called reflection coefficients for the AUTO case. In the AUTO case, they will all 
have a magnitude less than one, and represent reflection coefficients in an acoustic tube model of 
speech production. What meaning they may have in the COVAR case is not clear. 


3.4 Numerical Accuracy 


In essence, the solution is an indirect inversion of a matrix whose elements are the {c,,} coefficients, 
whose determinant is the product B)6,B.:-°-By_}. Theoretically, the coefficients {6,} are non- 
negative, and in fact are positive for the AUTO case. A negative value for one of the coefficients {B,} 
can only result from a numerical error. A zero value could result in the COVAR case. 


Any very small value of one of the coefficients {8,} will indicate problems, for they are used as divi- 
sors in Eq. (8b) for COVAR and as divisors in Eq. (10) for COVAR and AUTO, thus amplifying the 
effects of any numerical errors in evaluating the numerators of Eq. (8a) and Eq. (10). This problem is 
not relieved by resorting to other solution methods such as Gauss-Siedell, and can only be handled by 
increasing the number of significant figures, lowering the value of M, or by properly pre-emphasizing 
the data [1, Chapter 9]. 


Both subroutines have test statements to test for a zero or negative a, or B,. A print statement is 
placed at the appropriate point in the program to warn the user that a singular matrix has been defined. 
4. Program Description 
4.1 Usage 


Inputs and outputs to the subroutines COVAR and AUTO are in the argument lists, 
COVAR(N,X,M,A,ALPHA,GRC) and AUTO(N,X,M,A,ALPHA,RC). The inputs are 


N the number, WN, of data points, 
X(-) the input data with X(k) = x(k-—1) for k = 1,2,...,N, 
M the order, M, of the filter. 
The main outputs are the coefficients {a,} contained in the dimensioned array with 


A(1) = 1 
A(k) = a,_, for k =2,3,...M+1. 


An auxiliary set of outputs are the coefficients {k,,} with 


GRC(m) =k,, for m=1,2,....M  (COVAR) 
RC(m) =k, for m=1,2,....M@ (AUTO) 
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The dimensioned variables X(-), A(-), and GRC(-) or RC(-) are dimensioned by the calling pro- 
gram. The dimension of X(-) must be at least N, the dimension of A(-) must be at least M + 1, and 
the dimension of GRC(-) or RC(-) must be at least M. ALPHA defines the residual energy of Eq. 
(11). 


Table 1: Program Variables 


TEXT COVAR AUTO 
N N N 

x(n) X(n+1) X(n+41) 
M M M 

Ce A(i+1) A(i+1) 
a — ALPHA ALPHA 
Zs GRC(m) RC(m) 
GH ee) R(m+2-—/) 
Bn  BETA(m+1) ALPHA 
Y mi GAM os 

i B(-) A(:) 


Table 2: Minimum Dimensions for Variables 


DIMENSIONED MIN. DIM. MIN. DIM. 
VARIABLE FOR COVAR' FOR AUTO 


X(-) N N 
At) , M+1 M+1 
CC(-) M+1 

R(-) M+1 
GRC(-) M 

RC(-) M 
BETA(-) M 

B(-) M(M-1)/2 


4.2 Program Variables 


Table 1 shows a listing of the variables in this discussion. While some of the variables in the discus- 
sion have double subscripts, only singly dimensioned variables are used in the program. 


The coefficients {Dink} do not appear explicitly in AUTO, for they are related to the coefficients {an} 
by Eq. (9a). In COVAR these coefficients are stored in the singly dimensioned array B(-) in the order 


(11,521,629, 631,532, 633,b41,---by—1.y—1 - 


Table 2 shows the minimum dimension needed for each of the dimensioned variables. The subroutines 
are presently set to handle values of M up to 20. For values of M larger than 20, the dimension state- 
ments in the subroutines must be changed according to Table 2. 


5. Test Program 


A sample test program and its output is given. This particular program first generates a 40 point 
data sequence which is the unit sample response of the filter 


1/[1 — 1.4427! + 1.262-2 — 0.81273] . 


As the filter is third order and the numerator is of lower order than the denominator, the covariance 
approach with M=3 theoretically reproduces the denominator and gives a total squared error of zero. 
The test program gives the results of both covariance and autocorrelation methods for M=2 and M=3, 
and for M=3 indicates this, within the limits of computer accuracy, giving a residual energy of 0.6x10™> 
for M=3 as opposed to 1.7 for M=2. 
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The data sequence is then shifted by three steps, with three zeros appended at the start and end for 
the new 46 point sequence. Running both the methods on this sequence theoretically gives identical 
results to the autocorrelation method for the 40 point sequence with M=3. This is seen in the final 
results for the test program printout. 


References 
1. Markel, J. D. and Gray, A. H., Jr., Linear Prediction of Speech, Springer Verlag, New York, 1976. 


Table 3 


OUTPUT DATA FROM AUTO/COVAR TEST PROGRAM 


1 0.100000R 01 -0.891115E 00 
2 -0.108019E 01 0.197712E 00 
3 0.197712E 00 0.Q000000E 00 
.172095E 01 
1 0.100000E 01 -0.905548E 00 
2% -0.71619928. 91 0.283199E 00 
3 0.283199E 00 0.Q000000E 00 
308386E 01 
1 0.100000E 01 -0.935486E 00 
2. =0:..143999E 07 0.313706E 00 
3 0.125999F 01 -0.809997E 00 
4 -0.809997E 00 0.000000E 00 
619888E-05 
1 0.100000E 01 -0.905548E 00 
2. =6273 S0Sr 01 0.283199E 00 
3 0.110826E 01 -0.710041E 00 
4 -0.710041E 00 0.000000E 00 
152910E 01 
1 0.100000E 01 -0.905548E 00 
2 +8 .136308E 01 0.283199E 00 
3 0.110826E 01 -0.710042E 00 
4 -0.710042E 00 0.000000E 00 
» 15291 08-01 
1 0.100000E 01 -0.905548E 00 
2  =-0.136308E. 01 0.283199E 00 
3 0.110826E 01 -0.710041E 00 
4 -0.710041E 00 0.Q000000E 00 
- 1529T0E 01 
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1. Introduction and Purpose 


The lattice formulation for linear prediction was introduced in speech processing by Itakura. [1]. 
Recently, Makhoul [2] showed the existence of a class of such lattice methods all of which have the 
following properties in common: (1) windowing of the signal is not required; (2) the resulting all-pole 
linear prediction filter is guaranteed to be stable; (3) stability is less sensitive to finite wordlength com- 
putations; and (4) quantization of the lattice model parameters (for the purpose of data compression) 
can be accomplished within the recursion for retention of accuracy in representation. However, the 
computation for these lattice methods is about four times more expensive than the traditional auto- 
correlation and covariance methods. To overcome this drawback, Makhoul introduced the covariance 
lattice (CL) methods: these compute the lattice model parameters directly from the covariance of the 
signal, and thus require about the same order of computational complexity as the traditional autocorre- 
lation and covariance methods [2]. (For additional references on lattice methods, see [3]-[5].) 


Given the covariance matrix of the signal, the subroutines COVLAT and CLHARM described 
below compute the parameters of the lattice model, known as the reflection (or partial correlation) 
coefficients, and the predictor coefficients of the all-pole filter. COVLAT is a general routine in that it 
allows, through proper specification of input parameters, the choice of any one of a large class of CL 
methods. On the other hand, CLHARM uses a specific method, namely the harmonic-mean method 
[2]; consequently, the routine CLHARM requires less computation than COVLAT as it capitalizes on 
the specific structure of the harmonic-mean method. 


We also give two other subroutines, COVARI1 and COVAR2, for computing the covariance of the 
signal. When using COVARI, one computes at the outset the covariance of the signal up to the max- 
imum desired lag, and employs this fixed covariance in the estimation of all the reflection coefficients. 
COVAR?, on the other hand, recomputes for each new lattice stage the covariance up to that stage in a 
manner that makes maximum use of the signal data [2]. To allow the use of CL methods with either 
type of covariance computation, the subroutine COVLAT and CLHARM have been organized for com- 
puting the reflection coefficient of only one stage at a time. 


Covariance computation with COVARI1 is more common among speech researchers. It can be 
shown that when the routine COVAR2 for computing the covariance is used in conjunction with the 
harmonic-mean method, the results obtained are identical to those using Burg’s technique [3] as 
described in [6], the latter technique being more popular in the geophysics area. However, the use of 
the covariance lattice routine CLHARM given below results in about a four-fold savings in computa- 
tion. 


2. Brief Review of the Methods 


For a detailed exposition of covariance lattice methods, the reader is referred to [2]. The purpose of 
this review is to aid the reader in understanding the description of program parameters given in Section 
3 below, and to point out some changes in the algorithm that are necessitated by the ANSI FORTRAN 
convention of nonzero subscripts. 


* This work was supported by the Information Processing Techniques Branch of the Advanced Research Projects Agency. 


+ Partial correlation coefficients are actually the negative of the reflection coefficients as defined here. 
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2.1 Lattice Formulation 


Linear prediction methods model the signal spectrum by an all-pole spectrum with a transfer func- 
tion given by 


H(z) = G/A(z) (la) 


where 
A(z) =1+ ¥ art (1b) 
k=l 


is the inverse filter, G is a gain factor, a, are the predictor coefficients, and p is the number of poles or 
predictor coefficients in the model. If H(z) is stable (minimum phase), 4 (z) can be implemented as a 
lattice filter [1], as shown in Fig. 1. 


f.(n) 
: e(n) 


bp (n) 


Fig. 1 Lattice inverse filter 4 (z) 


The quantities f,,(m) and 6,,(n) denote the forward and backward errors or residuals at the output of 
the m” lattice Stage at time n. From Fig. 1, we have: 


foln) = bo(n) = s(n) (2a) 
fin Sn) = fn 1) + Kin 41 8 (n-D (2b) 
bnti(n) = Kathy (a) + b,(n—-D) (2c) 


where s(n) is the input signal, K,,+; is the reflection coefficient at stage m + 1, and e(n) = f,(n) is 
the output residual. 


The reflection coefficients K,, in the lattice are uniquely related to the predictor coefficients. We 
give below the recursive relations for obtaining a, from K,,. These relations are employed by both 
subroutines COVLAT and CLHARM: 


q im Zo 


m K 
q\m) — q (m0 eG 9 Pepe, (3) 


where a/”, 1 < j < m, are the predictor coefficients at stage m. The equations in (3) are computed 
recursively for m = 1,2.,...,p, the final solution is given by ihe a)”, 1 < j < p. The necessary and 
sufficient condition for the stability of H(z) is given by 


[Kb ky 2 ee eee (4) 
2.2 Covariance Lattice Recursions 


For easy reference, major relations used by the two subroutines COVLAT and CLHARM are given 
below, employing only nonzero subscripts. (For example, the covariance matrix element #(0,0) in [2] 
is denoted here by #(1, 1).) 


Assume that the input signal s(n) of the lattice in Fig. 1 is given in the range 1 <n <N. We 
then define the following four quantities: 
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pre Fw (5a) 
n=L 
N 

Bn = >, 62(n) (Sb) 
n=L 
N 

B= > b2(n-1) (Sc) 
n=L 
N 

Cn = YD fi (1) by (2-1). (5d) 
n=L 


The lower limit L = p + 1 if COVARI1 is used for the covariance computation, and L = m+1 if 
COVAR2 is employed instead. The time averages in Eq. (5) approximate expected values considered 
in reference [2], where the above quantities F,,, B,,, B,,, and C,, are denoted, respectively, by F,,(n), 
B,,(n), B,,(n—1) and C,,(n). Below, we shall call F,, and B,, as the mean-square value (or the vari- 
ance) of the forward and backward residuals, respectively. 


From Eqs. (2) and (5), one can show that the residual variances at stage m+l1 are given in terms of 
Fino Bns Cm and K,,41, as follows: 


Ba Koi Fg + 2B Cot Be. (6b) 
We employ the symbols K/,,, and K’,, to denote the reflection coefficient at stage m +1 produced 
by minimizing, respectively, the forward residual variance F,,,, (forward method), or the backward 


residual variance B,,,, (backward method). K,,,,; or K/,,, shall denote the r" mean between Kj ,, and 
K®,, (see below). K/and K® have the same sign [2], which shall be denoted by S: 


S = sign K/ = sign K°. 
2.2.1 General Covariance Lattice Method (COVLAT) 


Three quantities, F,,, B,, and C,,, are needed for computing the reflection coefficient at stage 
m +1. The three quantities are computed in the subroutine from the following equations: 


F,,= (1,1) +2 >» a o(1,i+1) + ss la”? (i +1, i+1) 


i=l i=] 


1 om 
425 Fao eet (7) 


B, = 6(m+2,m+42) +2 x a” &(m+2,m+2—i) 
i=] 
+ ¥ la ?b(m+2-i,m4+2-i) 
i=1 
m-l1 om 
+25 YY af™a)™ o(m+2-i,m+2-/) (8) 


i=1 j=i+l 


C, = 6(1,mt2) + ¥ a 16 (,m42—-1) + 641, m42)] 


i=] 


+ [a P26 G+1,m+2—) 


m-—1 m 
+ DY 96a" (oC +1,m+2—-/) + 6(j+1,m42-i)] (9) 
i=l j=it+l 
We then have 
i = Cl Br (10) 
Kha = — Cal Fr ap 
Khar = SIS (UK hal + [Kea IO, (12) 
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where r is the order of the generalized mean. The three specific choices r=0, —1, and —oo 
correspond to the three methods: geometric-mean method corresponding to that of Itakura [1], 
harmonic-mean method corresponding to that of Burg [3], and the minimum method [2]. The 
reflection coefficient estimates for the three methods are given below: 


eee ee = SV Kj i Kins (13) 
K-1, =-2C,,/(F,4+8,) (14) 
K-< = § min ¢K7Z¢ |,4)Ke 474) (15) 
Stability of the all-pole filter H(z) can be guaranteed if r < 0 [2]. Given F,,, B,,, C,, and the com- 


puted value for K,,,,, COVLAT computes the residual variance F,,,, and B,,, using the recursive 
relations (6). 


2.2.2 Harmonic-Mean Method (CLHARM) 


For this method, substantial savings can be achieved by computing the sum of the forward and 
.backward residual variances directly as follows: 
F. + B,, = 60,1) + 6(m42,m42) 
+2 ¥ af (oC, i+]) + 6(m+2,m+2-i)] 
i=| 


= > [a [6 CG41,i+1) + 6(m42—-i,m42-i)] 


i=] 


ol eee 
42D Y aa! (6G4+1j+D + 6(m4+2-i,m4+2-))] (16) 
i=l j=i+l 


The expression for C,, stays the same as given above in Eq. (9). Then, K,!, is computed using Eq. 
(14). 


For the special case of the harmonic-mean method, the value of K, 1, given in Eq. (14) minimizes 
the sum of the forward and backward variances F,,,,, + B,,4;. One can show from Eqs. (6) and (14) 
that 


Fa + By = ll — (Ki), 71, +B,,). (17) 


m 


Subroutine CLHARM uses Eq. (17) instead of Eq. (6). 
2.3 Covariance Computation (COVARI and COVAR2) 


Given the signal s(n), 1 < n < N, its covariance is computed using one of the two methods given 
below. 


2.3.1 Method 1 (COVARL) 


(i,j) = y s(k+l—D)sk 41-7), 14.4 7 & pt, (18) 


k=p+l 


where p is the order of the predictor or the number of lattice stages. Only the first row of p + 1 ele- 
ments (i=1, 1<j<p+l) of the covariance matrix is computed using the above expression. All other Dp 
rows are computed using the symmetry property of the covariance matrix #(i,/) = @(,i) and the fol- 
lowing recursive relations: 


(ij) = dGi-1,j-1) + s(pt+2—i) s(p4+2-/) 
— s(N+2-—i)s(N+2-/), 2< i 7 < pt. (19) 


The elements $(i,/) are computed only once and remain fixed throughout the solution. 
2.3.2 Method 2 (COVAR2) 


Here, all covariance terms up to stage m are recomputed for each new stage. The covariance for 
stage m is defined as follows: 


bij) = y skti-) Merl), ety = mt! (20) 


k=m+1 
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The first m rows of the covariance matrix are computed from the covariance for stage m — 1, using the 
symmetry property and the following recursive relations: 


bmn (id) = bm—-rCiJ) — s(m+1—-)s(m4+1-/), 1 <i igm (21) 
The elements of the (m+1)"" row are computed as follows: 
N 
d,(mtl,1)= SY slk)s(k—-m) (22a) 
k=m+1 
6, (mtl1,/) = d_-1(mj-l1) — s(N+1l—m)s(N+4+2-/), 2 <j < m1. (22b) 


2.4 Number of Computations 


For computing the (p+1)x(p+1) covariance matrix for an N-sample signal, the number of multi- 
plies and adds required by the programs COVAR]1 and COVAR2 are given in Table 1. 


Number of Multiplies 
Method and Adds 
COVARI N(p+1) 


COVAR2 


Table 1. Number of multiplies and adds required for the covariance computation. 


The total number of computations required for computing p reflection coefficients using COVLAT and 
CLHARM are given in Table 2. 


2 
COVLAT <p +2p += p = p+ 2p +p -9 
eo 2 pe eee oe 


Table 2. Number of computations for getting p reflection coefficients. 


Number of shift operations given in Table 2 refers to multiplication by 2. Computations given there 
include the evaluation of forward and backward errors (see Eq. (6)) for COVLAT (4p multiplies, p 
shifts and 4p adds) and the evaluation of forward-plus-backward error (see Eq. (17)) for CLHARM (2p 
multiplies and p adds). In counting the adds, we did not consider the adds that figure in the subscripts. 
In addition to the computations given in Table 2, use of COVLAT requires p evaluations of the func- 
tion GMEAN (Section 3.2), which is the generalized mean between the reflection coefficient values 
given by the forward method and the backward method (see Eq. (12)). 


The total number of computations required for computing the reflection coefficients from the signal 
data is then found by adding the number of computations for obtaining the signal covariance (Table 1) 
to that for computing the reflection coefficients (Table 2). 
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3. Program Description 
3.1 Main Program 


The main program listed in the Appendix is provided primarily to show how the subroutines are 
called and to produce results for two test cases. For any practical problem, the user should write his 
own main program. 


For each test example, the main program computes and prints three sets of reflection coefficients 
obtained from: (1) the harmonic-mean method using CLHARM, (2) the geometric-mean method using 
COVLAT with r = 0 and (3) the minimum method using COVLAT with r = —o. For case (1), the 
main program normalizes the forward-plus-backward error at the output of the last (p) stage, F, + B,, 
by dividing by 2¢(1,1) and prints this normalized error. The three CL methods are demonstrated 
using first Method 1 (COVAR1) for the covariance computation and then Method 2 (COVAR2). 


3.2 Subroutines 


(a) COVLAT(PHI,NMAX,M,R,IFLAG,A,K(M),FERROR,BERROR,SCR) 
Called from the main program; the parameters R and IFLAG define a class of CL methods. 


(b) GMEAN(KF,KB,R,IFLAG) 
Called by COVLAT; computes the generalized mean between the estimate KF of the reflection 
coefficient produced by the forward method and the estimate KB by the backward method. 


(c) CLHARM(PHI,NMAX,M,A,K(M),ERROR,SCR) 
Called from the main program; uses the harmonic-mean method. 


(d) COVARI1(SIG,NSIG,NSTAGE,PHI,NMAX) 
Called from the main program. Since the covariance computed by this method does not depend 
on the stage number, COVAR|I is called only once before computing the reflection coefficients in 
a stage-by-stage fashion. 


(e) COVAR2(SIG,NSIG,M,PHI,NMAX) 
Called from the main program. Since the covariance computed by this method depends on the 
lattice stage number, COVAR2 is called for each new stage before the reflection coefficient is 
computed. 


3.3 Description of Parameters 
PHI Covariance matrix 


NMAX Maximum dimension of PHI, or one plus the maximum number of lattice stages. (Actu- 
ally, PHI needs to have the dimension NSTAGE+1 only. However, the user may be 
interested in experimenting with different values for NSTAGE: for example, to drive the 
prediction error(s) sufficiently low, or for any other reason. NMAX in this case is then 
one plus the maximum value of NSTAGE that the user may want to consider. Notice 
that the.maximum dimension of each of the vectors A, K and SCR defined below may be 
set to NNMAX-—1; however, we have set those dimensions to NMAX for convenience.) 


M Lattice stage number 


NSTAGE Number of lattice stages or reflection coefficients, or the order of the all-pole model 
(NSTAGE is the same as p used in Section 2.) 


R Order of the generalized mean; filter stability is guaranteed for R < 0. 


IFLAG = —] means R = —o (Minimum method) 
= 1 means R = +o (Maximum method) 
= 0 means R is finite 


A A(1) through A(NSTAGE) are the predictor coefficients of the all-pole filter 
K Reflection coefficient vector (declared to be real) 
FERROR _ Forward error F,, at m=NSTAGE 
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BERROR _ Backward error B,, at m=NSTAGE 


SCR Scratch vector 

KF 7 Reflection coefficient given by the forward method (declared to be real) 

KB Reflection coefficient given by the backward method (declared to be real) 

ERROR Forward-plus-backward error, (F,,,+8B,,) at m=NSTAGE 

SIG SIG(1) through SIG(NSIG) are the signal samples 

NSIG Number of signal samples. (NSIG is the same as N used in Section 2.) 

NO Soe .Y number (This is defined in the main program through the integer FUNCTION 


3.4 DIMENSION Requirements 


The DIMENSION statement in the main program should be modified according to the requirements 
of each particular problem. The parameters included in the following DIMENSION statement conform 
to the above parameter definitions: 


DIMENSION PHI(NMAX,NMAX),A(NMAX),SCR(NMAX),SIG(NSIG) 
REAL K(NMAX) 


3.5 Output 


The main program prints out NSTAGE reflection coefficients and the errors FERROR, BERROR 
and ERROR, all defined above. 


3.6 Summary of User Requirements 
(a) Determine the value of NMAX and adjust DIMENSION statement. 


(b) (Optional) Specify a subroutine for quantizing each reflection coefficient as it is computed. (Com- 
ments are provided in COVLAT and CLHARM as to where to insert a call to the user’s own 
quantization routine. For details on quantization, refer to [7] and [8]. No modification of the 
subroutines COVLAT and CLHARM is necessary if the user’s application does not require quant- 
ization or if the user does not want to perform quantization within the recursion.) 


4. Test Problems 


The main program has two 4" order (i.e., NSTAGE=4) test problems: (1) Non-Toeplitz covariance 
matrix case; and (2) Stationary case where the covariance matrix reduces to a Toeplitz autocorrelation 
matrix. For the first problem, the main program defines a 32-sample signal vector SIG in data state- 
ments. This signal segment was taken from 10 kHz sampled real speech data for which the traditional 
covariance method produced an unstable filter. (The "reflection coefficient" values obtained using the 
covariance method were: —1.023, —0.168, —1.684, 1.03.) 


As seen from the listing of the main program output (Table 3), the CL methods produce stable 
filters. For the second problem, which requires a stationary (or windowed) signal, the main program 
appends NSTAGE zero samples on both sides of the above-mentioned 32-sample signal vector. This 
guarantees that the resulting covariance matrix is Toeplitz. For the stationary case, all CL methods 
Should give the same values for the reflection coefficients. Also, the forward error FERROR and the 
backward error BERROR should be equal. These results are readily verified from the output listing 
given in Table 3. 
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Table 3 


***FIRST TEST EXAMPLE. NON-TOEPLITZ COVARIANCE MATRIX CASEx** 
SOLUTION USING COVARIANCE MATRIX COMPUTED VIA METHOD 1 


REFLECTION COEFFICIENTS FROM PROGRAM CLHARM 
-0.98576966E+00 0. 75438073E+00 =-0.T65097/53E+00 0.77495836E+00 


FORWARD-PLUS-BACKWARD ERROR = 0.39345233E-02 


REFLECTION COEFFICIENTS FROM PROGRAM COVLAT (R=0, GEOMETRIC MEAN) 
-0.98578447E+00 0.75686465E+00 -0.16748354E+00 0. 792762312400 


FORWARD ERROR = 0.46530087E-02, BACKWARD ERROR = 0.31714242E-02 


REFLECTION COEFFS. FROM PROGRAM COVLAT (R=-INFINITY, MINIMUM METHOD) 
-0.980396711E+00 0.69331092E+00 -0.172956089E+00 0.60512754E+00 


FORWARD ERROR = 0.63612880E-02, BACKWARD ERROR = 0.37597384E-02 
SOLUTION USING COVARIANCE MATRIX COMPUTED VIA METHOD 2 


REFLECTION COEFFICIENTS FROM PROGRAM CLHARM 
-0.98586581E+00 0.48746297E+00 0.27865216E+00 0.26831956E+00 


FORWARD-PLUS-BACKWARD ERROR = 0.39385329E-01 


REFLECTION COEFFICIENTS FROM PROGRAM COVLAT (R=0, GEOMETRIC MEAN) 
-0.98589755E+00 0.50014503E+00 0.31662578E+00 0.35155942E+00 


FORWARD ERROR = 0.61631529E-01, BACKWARD ERROR = 0.17051948E-01 


REFLECTION COEFFS. FROM PROGRAM COVLAT (R=-INFINITY, MINIMUM METHOD) 
-0.97801873E+00 0.39491358E+00 0.18140856E+00 0.14482672E+00 


FORWARD ERROR = 0.65123382E-01, BACKWARD ERROR = 0.14874170E-01 
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Table 3 
(Continued) 


***SECOND TEST EXAMPLE. STATIONARY OR TOEPLITZ MATRIX CASE*** 
SOLUTION USING COVARIANCE MATRIX COMPUTED VIA METHOD 1 


REFLECTION COEFFICIENTS FROM PROGRAM CLHARM 


-0.97392271E+00 0.22170082E+00 0.20598015E+00 0.14759705E+00 


FORWARD-PLUS-BACKWARD ERROR = 0.45846863E-01 


REFLECTION COEFFICIENTS FROM PROGRAM COVLAT (R=0, GEOMETRIC MEAN) 
-0.97392271E+00 0.22170082E+00 0.20598015E+00 0.14759705E+00 


FORWARD ERROR = 0.45846863E-01, BACKWARD ERROR = 0.45846863E-01 


REFLECTION COEFFS. FROM PROGRAM COVLAT (R=-INFINITY, MINIMUM METHOD) 
-0.97392271E+00 0.22170082E+00 0.20598015E+00 0.14759705E+00 


FORWARD ERROR = 0.45846863E-01, BACKWARD ERROR = 0.45846863E-01 


SOLUTION USING COVARIANCE MATRIX COMPUTED VIA METHOD 2 


REFLECTION COEFFICIENTS FROM PROGRAM CLHARM 
-0.97392271E+00 0.22170082E+00 0.20598015E+00 0.14759705E+00 


FORWARD-PLUS-BACKWARD ERROR = 0.45846863E-01 


REFLECTION COEFFICIENTS FROM PROGRAM COVLAT (R=0, GEOMETRIC MEAN) 
-0.97392271E+00 0.22170082E+00 0.20598015E+00 0.14759705E+00 


FORWARD ERROR = 0.45846863E-01, BACKWARD ERROR = 0.45846863E-01 


REFLECTION COEFFS. FROM PROGRAM COVLAT (R=-INFINITY, MINIMUM METHOD) 
-0.97392271E+00 0.22170082E+00 0.20598015E+00 0.14759705E+00 


FORWARD ERROR = 0.45846863E-01, BACKWARD ERROR = 0.45846863E-01 
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and 


J. D. Markel 


Signal Technology, Inc. 
15 De La Guerra Street 
Santa Barbara, CA 93101 


1. Purpose 


The purpose of the subroutine LPTRN is to allow the user to transform between the different 
parameter sets often used in linear predictive speech analysis to describe the synthesis filter 


H(z) = Va/A(z), (1) 


where 


M 
A(z) = § az* with ag=1. (2) 
k=0 


The program is a slight modification of the program LPTRAN in [1], where a more detailed discussion 
may be found. 


2. Parameter Set Choices 


Table 1 shows the choices of parameter sets to be used as inputs, along with their FORTRAN labels. 
The column labeled I, where I = 1, 2,...,6, defines which parameter set is to be used as an input to the 
subroutine. All other parameter sets are then outputs. 


I Name Text FORTRAN Subscript Range 
1 filter coefficients ag= A(1) = 1 
a, A(j+l) =SAYU) jf =1,2,...M 

gain a ALPHA 

2 cepstral coefficients c(j) C(U+1) j=0,1,...,.M 

3 autocorrelation coefficients r(j) R(G+1) j=0,1,...,M@ 

4 reflection coefficients k; RCY) fa tl ae 
input energy r(0) R(1) 

5 log area ratios In(S/S;) ALARG) j =1,2,...M 
input energy r(0) R(1) 

6 area functions S; AREA (j+1) j=0,1,...,.M@ 
input energy r(0) R(1) 


Table 1. Definitions of Text and Fortran Variables for subroutine LPTRN 


In each case, there are a total of + 1 parameters which uniquely describe the filter H(z). When 
the array A(-) for the filter coefficients is used as an input (I=1), then the value of A(1) is ignored, 
since @)= 1 by definition. When A(-) is an output (I#1), then A(1) is set equal to one by the pro- 
gram. When the array representing the are functions, AREA(-), is used as an input (I=6), then M + 2 
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parameters are actually entered into the subroutine. When AREA(-) is an output (I~6), then 
AREA(M-+1) is always set equal to one for normalization. 

2.1 Filter Coefficients 


This is the parameter set {a ,@,...,a,4} defining the filter denominator as in Eq. (2). The coefficient 
ay equals one by definition. On input A(1) is ignored (I=1). When A(-) is an output (I¥1), then 
A(1) = 1. During the calculations another dimensioned array, SA( ) is used, where 


SAGY)=a; for j =1,2,...,M. 
This is to avoid changing any values of A(-) when it is used as an input (I=1). 
2.2 Cepstral Coefficients 


When H(z) is stable and causal, or A(z) is minimum phase, a set of cepstral coefficients {c(k)} is 
defined by 


IntVe/A(z)]= F clk)z-*, (3) 
k=0 
or equivalently, 
c(0) = InlVa] (4a) 
c(1) +a,=0 (4b) 
nc(n)+na,=-— e (n—k)c(n—k)a, , (4c) 
k=l 
for n= 2.,....M. 


Equation (4c) can also be used for n > M, provided one defines 
a. = 0 for k> me. (5) 


Equations (4) can be used directly to find the cepstral coefficients from the filter coefficients or vice 
versa. If the filter H(z) is not stable, the results are meaningless. 


2.3 Autocorrelation Coefficients 
When the filter H(z) is stable, the autocorrelation coefficients {r(k)} are defined by 
A(z)H(1/z)= SY r(k)z*, (6) 
and are even functions of k, 
r(k) =r(—k). (7) 


The autocorrelation coefficients {r (0),r(1),...,, (Z)} can be used as inputs or outputs in the subrou- 
tine. When used as inputs, the standard Levinson recursion is used to define a "step-up" procedure. 
This is abbreviated as follows. First there is an initialization of the form 


ag=r(0), k,;=—r(1)/r(0), (9a) 
aig= 1, ai = Ky, (9b) 
a, = ao(1—k ?). (9c) 


This is followed by the recursive steps 
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m-1 
kK, = | > Hm gs Pe ; (10a) 
j=0 
ano= 1 (10b) 
fn Ug aa Ot i = Ll (10c) 
Anm = Km > (10d) 
Qm = Am—-\A—-k,2) , (10e) 
for m = 2,3,...,M. At the completion of the recursion, 
Gp = Gigecoifot ick =D, (11a) 
and 
a=ay. (11b) 
A necessary and sufficient condition for stability of H(z) is that io 
2 eae aie ae as a ees (12) 


Therefore a division by zero cannot theoretically occur in Eq. (10a) for a stable filter. 


If the autocorrelation sequence is not an input (I¥3), then it is found from the same step-up 
approach, but with the input being the reflection coefficient set {k,k>,...,k,,} and the energy term r (0). 
The only changes in the formulas lie in replacing Eq. (9a) by 


ag=r(0), r(l)=—k,r(@) (13) 
removing Eq. (10a), and inserting at the end of step m the expression 
r(m) =— ¥& rlm—J)am, - (14) 
a 


2.4 Reflection Coefficients 


The reflection coefficients {k,,k>,...,k,y} represent the reflection coefficients for an equivalent acous- 
tic tube model for H(z). When used as inputs along with the filter energy 7 (0), the filter coefficients 
and autocorrelation coefficients are generated as described above. 


When the filter coefficients are used as an input (I=1), the reflection coefficients are generated 
through a "step-down" procedure as follows. First, the initialization of Eq. (11a) is performed. Then 
taking m = M,M-1....,1, 


ks, Res Amm ’ (15a) 
Qm-1,0 = ] ’ (15b) 
and 
Gin j — Km@mm—j 
am-1,j = ! ‘< <% : (15c) 


for j = 1,2,...,m—1. 
2.5 Log Area Ratios 
The log area ratios are a nonlinear transformation of the reflection coefficients, given by 
In(S;/S;-) = In[d—k,)/(+k,)] for j = 1,2,....M. (16) 


Scaling of all area functions by the same scale factor does not change the filter, for only the ratio of 
area functions enters into the calculations. When area functions are calculated as an output of the sub- 
routine (I#6) the value of Sj, is always normalized to one. 


2.6 Area Functions: 


The area functions {So,S;, ..., Sj} are areas of an acoustic tube model of H(z). They are related 
to the reflection coefficients by the equation 
S/S;-)= A-k)/A+k,) for j =1,2,..,.M. (17) 
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Scaling of all area functions by the same scale factor does not change the filter, for only the ratio of 
area functions enters into the calculations. When area functions are calculated as an output of the sub- 
routine (I#6) the value of S, is always normalized to one. 


3. Filter Stability 


The various transformations are meaningless if the filter H(z) is not a stable causal filter. Causality 
of the filter H(z) will be assumed, so that stability of H(z) is equivalent to H(z) being minimum 
phase. Stability can be guaranteed in advance for the cases I= 4, 5, and 6. When the reflection 
coefficients are input (I=4), their magnitudes must be less than one. When the log area functions are 
input (I=5) stability is guaranteed. When the area functions are input (I=6), they must all be positive 
to guarantee stability. 


When the parameters represent a stable H(z) there will be no error messages from the subroutine. 
Should parameters for an unstable H(z) be used then there will be attempts to take the logarithm of a 
nonpositive number in the evaluation of the log area functions, and there may be a division by zero. 


A short stability test, DO loop 330, is inserted in the subroutine to test the magnitudes of the 
reflection coefficients. If they do not satisfy Eq. (12) an error message is printed out. This subroutine 
provides a stability test for any rational digital filter whose denominator is expressed in the form of Eq. 
(2), and is equivalent to the Schur-Cohn method [2,3]. 


4. A Test Example 


The program can be used to test itself by changing the index I after each use. An example test pro- 
gram is given in the Appendix. Here the program generates the first parameter set to be used as an 
input (I=1) using M = 3, ap = 1, a; = —.45, a= .81, a3 = 0, and a = 10. By incrementing I from 1 
through 6, the roles of inputs are changed according to Table 1. Each printout will be the same, except 
for possible computer roundoff errors and the index I. A sample printout is shown in Table 2 for I = 1. 


The example used here illustrates how one can get additional information about autocorrelation 
values and cepstral coefficients. Note that the filter is actually only of second order. If M = 2 had 
been used, then the autocorrelation values and cepstral coefficients would only have been evaluated up 
to a subscript of 2. By setting M = 3 and a; = 0 they are evaluated up to a subscript of 3. 
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OUTPUT DATA FROM LPTRN TEST 


1 10.000001 


1.000000 
-0.450000 
0.810000 
0.000000 


2.302585 
0.450000 
-0.708750 
-0.334124 


2 9.999994 


1.000000 
-0.450000 
0.809999 
-0.000000 


2.302585 
0.450000 
-0.708750 
-0.334124 


3. 09..999999 


1.000000 
-0.449999 
0.809999 
-0.000000 


2.302584 
0.449999 
-0.708749 
-0.334124 


4 09.999999 


1.000000 
-0.449999 
0.809999 
-0.000000 


2.302584 
0.449999 
-0.708749 
-0.334124 


5 10.000011 


1.000000 
-0.449999 
0.809999 
-0.000000 


2.302586 
0.449999 
-0.708749 
-0.334124 


6 10.000011 


1.000000 
-0.449999 
0.809999 
-0.000000 


2.302586 
0.449999 
-0.708749 
-0.334124 


30. 
7 
=21 
=t5 


30. 
7 
64 
-15 


30. 
7 
~21 
et 


30. 
7 
~21 
=-15 


30. 
7 
-21. 
-15. 


Table 1 


PROGRAM 


993995 = 


- 705689 
-637573 
-978511 


.99329357 -- 
- 705680 
.637542 - 
-978488 


993957 = 


- 705680 
-637542 - 
.978488 


993957 .- 


. 705679 
63735425 = 
- 978483 


$3335; = 


- 705673 
-637538 
-978466 


$33957 = 


- 705673 


637538 = 
978466 


4.3-5 


0. 
-809999 


0 


-0. 
-000000 


0 


0. 
-809999 
0. 
-000000 


0 


0 


0.248618 
0.810000 
0. 

0.000000 


000000 


- 248618 
-809999 
-000000 
-000000 


- 248618 
-809999 
-000000 
-000000 


- 248618 
-809999 
-000000 
-000000 


248618 


000000 


248618 


000000 


- 507879 
-254057 
-000000 
-000000 


-507879 
- 254056 
-000000 
-000000 


- 507879 
-254055 
- 000000 
-000000 


- 507879 
-254055 
-000000 
-000000 


. 507879 
-254055 
-000000 
.000000 


- 507879 
-254055 
-000000 
-000000 


“oO OW “oO UW “oO Ul =“ - O UW -“- - OU 


-"-o OO UW 


- 732649 
- 5926313 
-000000 
-000000 


732645 


-526308 
-000000 
-000000 


. 732640 
- 526298 
. 999999 
-000000 


- 732640 
- 526298 
- 999999 
-000000 


- 732637 
«326290 
-999999 
-000000 


- 732637 
~526290 
«999999 
-000000 
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CHAPTER 5 


FIR Filter Design and Synthesis 


J. H. McClellan 


Introduction 


The problem of designing finite impulse response (FIR) digital filters experienced great activity in 
the early 1970’s. Most of this work was directed at the problem of optimal (in the weighted Chebyshev 
sense) filter design. The first program in this section, by McClellan, Parks, and Rabiner, will design the 
optimal Chebyshev approximation for linear phase filters. The Chebyshev optimization is done very 
efficiently by means of the Remez exchange algorithm, and this particular FIR filter design program has 
found wide use. Interestingly enough, this usage seems to have come as such from the designers of 
CCD (charge-coupled device) and SAW (surface acoustic wave) filters as from the digital filtering com- 
munity. Of course, any FIR design program can be used for these other problems since the underlying 
mathematics is the same. 


The second program in this section implements the classical method of windowed linear-phase filter 
design. Seven window types can be specified including Chebyshev and Kaiser windows. The third pro- 
gram designs maximally flat linear-phase lowpass filters. In both of these programs (5.2 and 5.3) the 
design can be expressed in closed form; no optimization is required. 


The final program, by Heute, is a subroutine for the synthesis of a finite wordlength (fixed-point 
rounding of coefficients) realization of a linear phase FIR filter. The program determines the minimum 
possible wordlength to meet the prescribed filter specifications. This technique requires, as a starting 
point, an exact (infinite-precision) design as would be obtained from a program such as the Remez 
exchange algorithm. Program 5.4 then seeks to fulfill the given specs. As a final comment, we note 
that the problem of optimizing the finite wordlength design of FIR digital filters remains an unsolved 
problem. 


The four programs contained in this section are but some of the programs (and techniques) that 
have been proposed for the design and implementation of linear-phase FIR digital filters. Other 
methods for Chebyshev approximation of linear-phase filters include Linear Programming [1,7]. 
Herrmann and Hofstetter’s method [2,5], and McCallig’s CONRIP program [6]. Among other error 
norms, the weighted least-squares norm is a straightforward method that is not restricted to linear- 
phase filters. 


One important difference among various methods is the manner in which the desired filter is 
specified. The filter design process is always a compromise among filter length, transition width, and 
passband and stopband deviation (in the bandpass case). Not all of these specs can be chosen arbi- 
trarily. In connection with program 5.1 the work of Herrmann, Rabiner and Chan [4] and Rabiner [8], 
describes tradeoff relationships among the various parameters. Similar results are available for window- 
ing and the maximally flat designs. 
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The design of nonlinear phase FIR digital filters has proven to be a more difficult problem. A par- 


ticular case of interest is the design of minimum phase filters. A polynomial rooting technique pro- 
posed by Herrmann and Schuessler [3] can be used to convert a linear-phase filter to a minimum phase 
design. The more general problem of optimal magnitude and phase approximation still remains as an 
unsolved problem. 
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FIR Linear Phase Filter Design Program 


J. H. McClellan 


Dept. EECS 
M.I1.T. 
Cambridge, MA 02139 


T. W. Parks 


EE Dept. 
Rice University 
Houston, TX 77001 


L. R. Rabiner 


Acoustics Research Dept. 
Bell Laboratories 
Murray Hill, NJ 07974 


1. Purpose 


This program uses the Remez exchange algorithm [1], [2] to design linear phase FIR digital filters with 
minimum weighted Chebyshev error in approximating a desired ideal frequency response. A unified 
treatment of the theory of this method of filter approximation can be found in Refs. [3]-[7]. The pro- 
gram has a special built-in section for specifying the more common ideal filter types such as multi-band, 
bandpass filters, Hilbert transform filters, and differentiators. It is also possible to approximate an arbi- 
trary ideal frequency response specified by the user, but two user-written subroutines must be provided 
to replace two of the existing subroutines [6]. 


2. Method 


The frequency response of an FIR digital filter with an N-point impulse response {h(n),n=0,1,...,N—1} 
is the z-transform of the sequence evaluated on the unit circle, i.e., 


N-1 
H(f)! = H(z)|,_ oir = > h(nje PF" . (1) 
n=0 
The frequency responses of a linear phase filter can be written as 
H(f) = G(Nexp] (= — (Ayam (2) 


where G(f) is a real valued function and L = 0 or 1. It is possible to show that there are exactly four 
cases of linear phase FIR filters to consider [3,5]. These four cases differ in the length of the impulse 
response (even or odd) and the symmetry of the impulse response [positive (L=0) or negative (L=1)]. 
By positive symmetry we mean h(n) = h(N—1—n), and by negative symmetry h(n) = —h(N-—1—n). 


In all cases, the real-valued function G(/) will be used to approximate the desired ideal magnitude 
specifications since the linear phase term in Eq. (2) has no effect on the magnitude response of the 
filter. The form of G(/) depends on which of the four cases is being used. Using the appropriate 


1 For convenience, throughout this paper the notation H(/) rather than H(e27/) is used to denote the frequency response of 
the digital filter. 
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symmetry relations, G(/) can be expressed as follows: 


Case 1: Positive symmetry, odd length: 


Nl 
G(f) = ¥& a(n) cos(2anf) (3) 
n=0 


where N1 = (N—1)/2, a(O) = A(N1), and a(n) = 2h(N1—n) for n = 1,2....,N1. 


Case 2: Positive symmetry, even length: 


Nl 
G(f) = ¥ b(n) cosl2a(n - 1 (4) 


n= 
where N1 = N/2 and b(n) = 2h(N1—n) for n = 1.,,...,N1. 


Case 3: Negative symmetry, odd length: 
Nl 


G(f) = ¥& c(n)sinQrnf) (S) 


n= 


where N1 = (N—1)/2 and c(n) = 2h(N1—n) for n = 1,2,...,N1 and h(N1) = 0. 


Case 4: Negative symmetry, even length: 
A 


z: 
G() = ¥ d(n)sina(n - DA (6) 
=I 


where N1 = N/2 and d(n) = 2h(N1—n) for n = 1,...,N1. 
All four cases can be combined into one algorithm. This is accomplished by noting that G(/) can be 
rewritten as G(/) = O(/) P(/) where P(/) is a linear combination of cosine functions. 


It is convenient to express the summations in (4)-(6) as a sum of cosines directly. Simple manipula- 
tions of (4)-(6) yield the expressions 


Case 2: 
NI | 1 Ni-1 _ 
b(n) cos[27(n — asi =cos(rf) } b(n)cos(2anf) (7) 
n=] n=0 
Case 3: 
NI N1-1 | 
> c(n)sinQanf) = sinQaf) SY t(n)cos(2rnf) (8) 
n=] n=0 
Case 4: 
NI iu 1 Mint 
d(n)sin[27(n — sl =sin(rf) } d(n)cos(27n/f) (9) 
n=] =0 


where 


b(1) = 6(0) + = 6) 


b(n) = + [b(n=1) + 5(n)], 
Case 2: Hla ae, a (10) 
b(N1) = > B(NI-1) 


Section 5.1 EQFIR 


o()) = 210) = > @(2) 


oe ; Gan ~ ce+1. 
Case 3: n =2,3.,...,N1 — 2 (11) 
thay & ; =(N1—2) 


cul > =(N1—1) 


a(1) = 4) - + a) 


tn + [d(n—1)—-d(n) 1, 
Case 4: Pas ONES 1 
ee > d(N1-1). 


(12) 


The motivation for rewriting the four cases in a common form is that a single central computation rou- 
tine (based on the Remez exchange method) can be used to calculate the best approximation in each of 
the four cases. This is accomplished by modifying both the desired magnitude function and the weight- 
ing function to formulate a new equivalent approximation problem. 


The original approximation problem can be stated as follows: given a desired magnitude response D(f) 
and a positive weight function W(/), both of which are continuous on a compact subset of 0,1 (note 


that the sampling frequency is 1.0) and given one of the four cases of linear phase filters li.e., the 
forms of G(/)], then one wishes to minimize the maximum absolute weighted error, defined as 


|JE|| = max W(f)|D(N-G(/)| (13) 


over the set of coefficients of G(/). 
The error function E(/) can be rewritten in the form 


E() = W(NID(N-G/1 = WNa/ ran _ P(f) 


if one is careful to omit those endpoint(s) where Q(/) =0. Letting D(f) = D(f)/O(/) and 
W(f) = W(S) OVC), an equivalent approximation problem is to minimize the quantity 


EW || = max W(f)|D()-P(/)| (15) 


(14) 


by choice of the coefficients of P(f). The set F has been replaced by F’ = F - {endpoints where 
O(f)=0}. 


The net effect of this reformulation of the problem is a unification of the four cases of linear phase FIR 
filters from the point of view of the approximation problem. Furthermore, Eq. (15) provides a 
simplified viewpoint from which it is easy to see the necessary and sufficient conditions that must be 
satisfied by the best approximation. Finally, Eq. (15) shows how to calculate this best approximation 
using an algorithm which can do only cosine approximations (Case 1). The necessary and sufficient 
conditions for this best approximation are given in the following alternation theorem [1]. 
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Alternation Theorem: If P(f) is a linear combination of r cosine functions i.e., 
r—1 
P(f) = ¥ aln)cos (Qanf) , 
n=0 


then a necessary and sufficient condition that P(/) be the unique best weighted Chebyshev approxima- 
tion to a continuous function D(f) on F' is that the weighted error function E(F) = W(/) 
[D(f)—P(J/)] exhibit at least r + 1 extremal frequencies in F’. 


These extremal frequencies are a set of r+1 points {fF}, i=1,2,...7.+1 such that 
Fi< Fy -+: <F, < Fyi, with ECF) =—E(F4)), i = 1,2.,...,rand |E(F)| = max|E(/)|. 


An algorithm can now be designed to make the error function of the filter satisfy the set of necessary 
and sufficient conditions for optimality as stated in the alternation theorem. The next section describes 
such an algorithm along with details of its implementation. 


3. Program Description 


The program consists of a main program that handles the input, sets up the appropriate approximation 
problem, calls the Remez exchange subroutine and handles the output of the optimal filter coefficients 
and the value of the minimum error. The program contains seven subroutines the most important of 
which are REMEZ, EFF and WATE. Subroutine REMEZ implements the Remez exchange algorithm 
for Chebyshev approximation. Subroutines EFF and WATE define the desired ideal frequency 
response, D(f), and the weight function, W(f), respectively. It is possible for the user to modify 
these subroutines to suit his own choice of ideal frequency response and tolerance scheme [6]. Figure 
1 provides a summary of the overall program structure. Note that the program yields the impulse 
response of the optimum filter, but does not produce a frequency response plot. Such a plot can be 
generated easily by the user with an FFT subroutine and some straightforward graphical programs. 


INPUT FILTER 
SPECS 
FILTER] TYPE 
BANDPASS DIFFERENTIATOR HILBERT 


TRANSFORM 


DENSE GRID FOR 
THE SET F 


WATE EFF 
WEIGHT FUNCTION DESIRED MAGNITUDE 
FORMULATE EQUIVALENT 
APPROXIMATION PROBLEM 
INITIAL GUESS FOR r+1 
EXTREMAL FREQUENCIES 


REMEZ 
SOLVE THE 
APPROXIMATION PROBLEM 


CALCULATE IMPULSE 
RESPONSE 


PRINT OUT THE 
OPTIMAL ERROR & 
IMPULSE RESPONSE 


Fig. 1 Overall flowchart of filter design algorithm. 
The specification of the filter design problem requires the following input parameters: 


NFILT The filter length in samples. NFILT must satisfy 3 < NFILT < NFMAX (the upper 
limit can be changed by the programmer). In the present implementation NFMAX 
has been arbitrarily chosen to be 128. 


JTYPE The type of filter 
(a) Multiple passband/stopband (JTYPE=1) 
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(b) Differentiator (JTYPE=2) 
(c) Hilbert transformer (JTYPE=3). 


EDGE An array of size 20. The frequency bands, specified by upper and lower cutoff fre- 
quencies, up to a maximum of 10 bands. 


FX An array of size 10. The desired frequency response in each band. 
WTX An array of size 10. A positive weight function in each band. 


LGRID The grid density, set to be 16 unless altered by the user by specifying a positive value 
for LGRID. 


The EDGE array specifies the set F to be of the form F = UB, where each frequency band B; is a 
closed subinterval of the frequency axis [0,1/2]. The arrays FX and WTX then specify the ideal 
response and weight function in each band. These input arrays are interpreted in different manners by 
the program depending on whether the filter type is a differentiator (JTYPE=2) or not. In the case of 
a differentiator FX(I) specifies the slope of D(/) in the frequency band B, while WTX(I) gives W(f) as 
1/WTX (1), resulting in a re/ative error tolerance scheme [6]. 


The set F must be replaced by a finite set of points for implementation on a computer. A dense grid of 
points is used with the spacing between points being 0.5/(LGRID-r) where r is the number of cosine 
basis functions. Both D(/) and W(/) are evaluated on the grid and stored in the arrays DES and WT 
by the subroutines EFF and WATE, respectively. Then the equivalent approximation problem is set up 
by forming D(f) and W(/) as in Eq. (14), and an initial guess of the extremal frequencies is made by 
taking r + 1 equally spaced frequency values. The subroutine REMEZ is called to perform the calcula- 
tion of the best approximation for the equivalent problem. The mechanics of the Remez algorithm will 
not be discussed here since they are treated elsewhere for the particular case of low-pass filters [4]. 


The appropriate equations ((3)-(12)) are used to recover the impulse response from the coefficients of 
the best cosine approximation obtained in the REMEZ subroutine. The outputs of the program are the 
coefficients of the best impulse response, the optimal error (min||E(/)||), and the r + 1 extremal fre- 
quencies where E(f) = + ||E(/)||. 


It is possible that one might want to design a filter to approximate a magnitude specification which is 
not included among the possibilities given above, or change the weight function to get a new tolerance 
scheme. In such cases, the user must code new functions EFF and WATE to calculate D(f) and 
W(f) as a function of f£ Note that the value of fis passed to each of these functions via the parame- 
ter FREQ. The input is the same as before, except that there are only two types of filters, depending 
on whether the impulse symmetry is positive or negative (JTYPE=1 or 2). One example of the design 
of a nonstandard filter using this program is given in Ref. [6]. 


4. Design Examples 


This design example illustrates the multiband capability of the program, for an N = 55 five-band filter 
with three stopbands and two passbands. The weighting in each of the stopbands is different, making 
the peak approximation error differ in each of these bands. The required input for this example is 
given in Table 1. Table 2 shows the computer output listing, and Figure 2 shows a plot of the filter fre- 
quency response on a log magnitude scale. 


An extensive set of design examples which illustrate the versatility of this program is given in Ref. [6]. 
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MULTIBAND FILTER 
N=55 


LOG MAGNITUDE IN dB 


O 0. 0.2 0.3 0.4 0.5 
FREQUENCY 


Fig. 2 Log magnitude response for an N = 55 multiband filter. 
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DEVIATION 
DEVIATION 
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DEVIATION 


Table 2 Output listing for an N = 55 multiband filter. 


UR eas | ae ey. | 0 ee | Pa 
l 
oO 


.000734754 
.006315947 
.021567374 
.026203127 
.032680369 
.034435446 
.034448378 
.034448593 


HHH HHH HH HEKE HHH HHH KKH HK HEHEHE KH KKH KEKHHKKHHHKHKEKKKEKEKEKEKKEKKEKKKEKKEKEKKEKRKEEE 


FINITE IMPULSE RESPONSE (FIR) 


LINEAR PHASE DIGITAL FILTER DESIGN 


REMEZ EXCHANGE ALGORITHM 
BANDPASS FILTER 


1) 


FILTER LENGTH = 
**%**x*%* IMPULSE RESPONSE *«**** 


0 


-10662652E-02 
0. 
-35755609E-02 
.90677854E-02 
-90906978E-02 
-29155630E-02 
-39637965E-02 
4141 72Z05TE- 01 
-11646759E-01 
.99630785E-02 
-92384245E-02 
.20406392E-01 
-19460483E-01 
. 31243014E-01 
-63045568E-02 


63777615E-02 


5 


5 


H ( 
H ( 
H ( 
H ( 
H ( 
H ( 
H ( 
H ( 
H ( 
H ( 
H ( 
H ( 
H ( 
H ( 


55) 
54) 
53) 
52) 
51) 
50) 
49) 
48) 
47) 
46) 
45) 
44) 
43) 
42) 


LOWER BAND EDGE 
UPPER BAND EDGE 
DESIRED VALUE 
WEIGHTING 
DEVIATION 
DEVIATION IN DB 


LOWER BAND EDGE 
UPPER BAND EDGE 
DESIRED VALUE 
WEIGHTING 
DEVIATION 
DEVIATION IN DB 


. 2048280 
0.6574051 
. 1420212 
0.4195698 


0.3474480 
-1713883 
0.7405902 


. 1031742 
0.2571672 


[eth | em | FR | i | an A) Pm ee | | Hm A Pea a) | PR | | ML A | | aa | CT Ie | TAR a 
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BAND 1 
.0500000 


.0000000 
.0034449 
se06a703 


BAND 5 
-4100000 
. 5000000 


. 0000000 
.0017224 
-55.2771702 


EXTREMAL FREQUENCIES--MAXIMA 


-1000000 
. 1800000 
. 2436160 
~2002232 
-4457143 


ic © OO. 


CS °C) G6) > 


.0167411 
-1089286 
. 1855804 
-2500000 
. 3600000 
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o:0 © oo © 


0.35784266E-01 


0.71496359E-01 
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1E 00 
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30) 
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BAND 2 


-1000000 
. 1500000 
-0000000 
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OF THE ERROR CURVE 
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BAND 3 


-1800000 
. 2500000 


.0000000 
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SO One a 


.0500000 
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FIR Windowed Filter Design Program - WINDOW 


L. R. Rabiner and C. A: McGonegal 
Acoustics Research Dept. 
Bell Laboratories 
Murray Hill, NJ 07974 
D. Paul 
MIT Lincoln Laboratory 
Lexington, MA 02173 
1. Purpose 


This program can be used to design FIR digital filters using the window method [1-3]. The program 
is capable of designing lowpass, bandpass, bandstop, and highpass filters for both even and odd values 
of N, (the impulse response duration in samples), using either a rectangular, a triangular, a Hamming, 
a Hanning, a Chebyshev, or a Kaiser window. 

2. Method 


This program uses the well known method of window design for FIR digital filters. If we denote the 
N-point windows as w(n), for 0 < n < N — 1, and we denote the impulse response of the ideal digital 
filter (obtained as the inverse Fourier transform of the ideal frequency response of the filter) as A (n), 
—co <n < oo, then the windowed digital filter is given as 


h(n)=w(n)h(n) O<n<N-1 
= otherwise (1) 


In the discussion above it is assumed that 4(n) incorporates an ideal delay of (N—1)/2 samples, and 
that w(n) is symmetric around the point (N—1)/2. 


In order to design windowed digital filters requires specification of 
1. Type of ideal filter - i.e., lowpass, highpass, bandpass or bandstop filter. 
2. Filter cutoff frequencies. 
3. Type of window - i.e., rectangular, triangular, Hamming, Hanning, Chebyshev, Kaiser. 
4. Window duration in samples. 


From the filter specifications the sequences 4 (n) and w(n) of Eq. (1) are computed, and the windowed 
filter is obtained as the final output. 


2.1 Form of the Windows 
The seven windows used in this program are 

1. Rectangular Window 

Triangular Window 

Hamming Window 

Hanning Window 

Generalized Hamming Window 

Kaiser (I,-Sinh) Window 

Chebyshev Window 


A 
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The form of each of these windows (assuming they are symmetric around n=0) is: 


Rectangular Window | 
For N odd win) ='1 —(N-1)/2 < n < (N-1)/2 (2a) 


For N even, w(n) = 1 —(N/2) < n < (N/2-1) (2b) 


Triangular Window 
For N odd, w(n) = 1-— |2n|/(N+1) —(N—1)/2°< n < (N-1)/2 (3a) 


For N even, w(n) =1— |2n+1|/N —(N/2) < n < (N/2-1) (3b) 


Hamming Window 


For N odd, w(n) = 0.54 + 0.46 cos[27n/(N-1)] —(N-1)/2 <n < (N-1)/2_~— (4a) 
For Neven, w(n) = 0.54 + 0.46 cos ee —(N/2) < n < (N/2-1) (4b) 
Hanning Window 
For N odd, w(n) = 0.5 + 0.5 cos[27n/(N+1)] —(N-1)/2 < n < (N-1)/2 (Sa) 
E Qn (2n +1) a * 
For Neven, w(n) =0.5+ 0.5 cos N41) (N/2) < n < (N/2-1) (Sb) 
Generalized Hamming Window 
For N odd, w(n) =a + (1—a)cos [27n/(N-1)] SAN 21)72 =& nv < (V—1)/2 (6a) 
For Neven, w(n) =a + (1—a)cos eee —(N/2) <n < (N/2-1) (6b) 
where a is variable parameter. 
Kaiser (\o-Sinh) Window 
Ie) = | 
For N odd, w(n) = —(N-1)/2 < n < (N—-1)/2 (7a) 
fpf ear 
For N even, w(n) = 2 eee —(N/2) a nN = (N/2-1) (7b) 


where @ is window parameter related to desired minimum stopband attenuation [4]. 

Chebyshev Window 

w(n) is obtained as the inverse DFT of the Chebyshev polynomial, evaluated at N equally spaced fre- 
quencies around the unit circle [2,3]. The parameters of the Chebyshev window are the ripple, 6,, the 


filter length, N, and the normalized transition width, AF. Figure 1 shows a plot of the frequency 
response of a Chebyshev window illustrating how 6, and AF are measured. 
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Fig. 1 Definition of Chebyshev window parameters in the frequency domain. 


Only 2 of the 3 parameters N, 6, and AF can be independently specified. If the parameters 5, and 
AF are specified, then N is determined by the equation 


i+6,, 
cosh7! | ——4 
Nilo f (8) 
es ae ee 
cos(7AF) 
If the parameters N and 6 » are specified, then AF is determined as 
AF = = cos’! ee ee eee (9) 
= ai, 
cosh— 
5, 
cosh oo 
Finally if the parameters N and AF are specified, then 8 » 1s determined as 
i, = a eee ee ee ee (10) 
l 
h|(NV—1 Se || 
COs | )cosh pee 


3. Program Description 
3.1 Usage 

The program consists of a main program and eleven subroutines. The main program reads the input 
data (i.e., the filter parameters discussed above), and the program prints the filter parameters, the win- 


dow coefficients, and the resulting impulse response of the digital filter (stored in array G). User 
prompts are given to aid in the input stage of the program. 


3.2 Description of Input Parameters 


CARD 1: NEAITYPE, JTYPE 
FORMAT: (14,12,12) 


NF — The filter length in samples. NF must satisfy the relation 3 < NF < 1024. For 
highpass or bandpass filters NF must be odd. 


ITYPE — The window type. Values of ITYPE from 1 to 7 correspond to rectangular, triangular, 


Hamming, generalized Hamming, Hanning, Kaiser, and Chebyshev windows, respec- 
tively. 


JTYPE — The filter type. Values of JTYPE from 1 to 4 correspond to LP (lowpass), HP 
(highpass), BP (bandpass), and BS (bandstop) designs, respectively. 
HST YPE = L-orJTYPE = 2. CARD 2: FC 
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If JTYPE = 3 or JTYPE = 4, CARD 2: FL, FH 
FORMAT: (2F 14.7) 


FC — Normalized filter cutoff frequency for lowpass and highpass filters. FC is normalized to 


the range 0 < FC < 0.5. (See Figure 2). 
IDEAL LOWPASS FILTER 


4 


(a) 


O FC 0.5 


IDEAL HIGHPASS FILTER 


0 FC 0.5 
Fig. 2 Definition of normalized cutoff frequency, FC, for (a) an ideal lowpass and, (b) an ideal highpass filter. 


FL,FH — Normalized filter cutoff frequencies for bandpass filters and bandstop filters. FL and 


FH are normalized so that 0 < FL < FH < 0.5. (See Figure 3). 
IDEAL BANDPASS FILTER 


0 re FH 0.5 


IDEAL BANDSTOP FILTER 


(b) 


°6 FL FH 0.5 


Fig. 3 Definition of normalized cutoff frequencies, FL and FH, for (a) an ideal bandpass, and, (b) an ideal bandstop filter. 


If ITYPE = 4, CARD 3: ALPHA 
If ITYPE = 6, CARD 3: ATT 
If ITYPE = 7, CARD 3: DPLOG,DF 


FORMAT: (2F 14.7) 
ALPHA — Parameter of the generalized Hamming window. 


ATT —__ The desired minimum stopband attenuation in dB (from which the parameter B of the 
Kaiser window is determined). 


DPLOG,DF — Ripple and transition width parameters for Chebyshev window designs. DPLOG 
is the desired filter ripple (on a decibel scale) and DF is the normalized transition 
width of the filter (0<DF<0.5). For Chebyshev windows exactly 2 of the 3 parame- 
ters NF,DP, and DF must be specified. The unspecified parameter (input set to 0) is 
determined analytically. 


CARD 4: IWIN 
FORMAT: (11) 
IWIN — _ IWIN is set to 1 to print w(n). 
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CARD 5: IMD 

FORMAT: (11) 

IMD — _ IMD is set to 1 to design a new filter, IMD is set to 0 to terminate the program. 
3.3 Description of Main Program (WINDOW) and Subroutines 


Main Program (WINDOW) — This program reads in filter parameters, determines the window and 
the ideal lowpass filter response, computes the windowed digital filter coefficients, transforms the 
lowpass filter to the specified filter type, writes out the filter coefficients and determines filter bandedges 
and ripples within each band. As an option it will also write out the window coefficients. 


TRIANG — This subroutines creates a triangular window. 

HAMMIN — __ This subroutine creates a generalized Hamming window. 

KAISER — This subroutine creates a Kaiser window. 

INO — This function evaluates J,(x) [4]. 

CHEBC — This subroutine computes the unspecified parameter of a Chebyshev window. 
CHEBY — This subroutine creates a Chebyshev window. . 

FLCHAR — This subroutine computes a high resolution frequency response of the resulting 


filter using a DFT algorithm. For all window designs (except the triangular win- 
dows for which no ripple or bandedges are readily obtained) each ideal filter band 
is searched for the peak ripple and the bandedge. The bandedge is defined as the 
point where the frequency response crosses the ripple bound. A more precise 
estimate of the bandedge is determined by linear interpolation of the frequency 
response of the filter in the vicinity of the bandedge. Finally the ripple (e., 
peak deviation from the ideal response) is determined as 


Ripple = 20 log (1+8,) GB) for passbands 
Ripple = 20 log (8,) (dB) for stopbands. 


RIPPLE — This subroutine determines the filter ripples and bandedges from the filter param- 
eters and the high resolution frequency response. 


COSH — This function evaluates cosh (x). 
COSHIN — This function evaluates cosh~!(x). 
ARCCOS — ___ This function evaluates cos™!(x). 


3.4 Dimension Requirements 


The dimensioned arrays are 


WwW — This array holds half of the window. (The second half is obtained by symmetry.) The 
size of W must be (NF+1)/2. 

G — This array holds half the filter coefficients. (Symmetry is again used for the second 
half.) Its size must also be (NF+1)/2. 

PR — This array is only dimensioned in the subroutine CHEBY. It holds the real part of the 
DFT of the window. The size must be NF. 

PI — This array is only dimensioned in the subroutine CHEBY. It holds the imaginary part 


of the DFT of the window. The size must be NF. 


RESP — _ This array is only dimensioned in the subroutine FLCHAR. It holds the high resolu- 
tion frequency response of the filter. The size of RESP is 2048 points. 
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3.5 Summary of User Requirements 


(1) Specify filter parameters NF, ITYPE, JTYPE, FC (or FL,FH), and special window parameters 
DPLOG, DF (for Chebyshev windows), ALPHA (for generalized Hamming windows) and ATT 
(for Kaiser windows). 


(2) Design w(n), h(n) and h(n) using the main program which prints h(n), the filter coefficients, 
which are in array G, and optionally prints w(n) in array W. 


4. Test Problems 


The input data for a set of 7 windowed filter designs which exercises some of the types of windows 
and filters is given in Table 1. The output for each of these 7 cases is given in Tables 2-8, respectively. 
Test example 1 is a 25 point lowpass filter with ideal cutoff frequency of 0.3. A rectangular window is 
used for this case. Example 2 is the same as example 1 except a Hamming window is used, and the 
value of NF is increased to 30. Example 3 is a 25 point lowpass filter with ideal cutoff frequency of 0.3. 
This case uses a triangular window. Example 4 is a 56 point bandpass filter designed using a Kaiser 
window. The ideal bandedges of the filter are 0.15 and 0.35, and the window is designed for 60 dB 
minimum attenuation. Example 5 is a 55 point bandpass filter designed using a Chebyshev window. 
The ideal bandedges are 0.15 and 0.35, and the peak stopband ripple is specified to be 60 dB (0.001 on 
a linear scale). The transition width of the window is unspecified (set to 0.) as this is the free parame- 
ter of the Chebyshev window for this run. Example 6 is a bandstop filter designed using a Chebyshev 
window. The filter length is unspecified (set to 0) and is determined from the specified ripple (60 dB) 
and transition width (0.05) for the window. The bandedges of the filter are 0.15 and 0.35. The last 
example is a 55 point highpass filter designed using a Kaiser window. The minimum stopband attenua- 
tion is 60 dB, and the ideal cutoff frequency of the filter is 0.35. Figures 4 to 7 show plots of the win- 
dow (part a), the window frequency response (part b), the filter impulse response (part c) and the filter 
frequency response (part d) for examples 2, 5, 6, and 7. 
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Fig. 4 Responses of the window (parts a and b) and the Fig. 5 Responses of the window (parts a and b) and the 
resulting filter (parts c and d) for example 2. resulting filter (parts c and d) for example S. 
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Fig. 6 Responses of the window (parts a and b) and the Fig. 7 Responses of the window (parts a and b) and the 
resulting filter (parts c and d) for example 6. resulting filter (parts c and d) for example 7. 


The listing of the main program and subroutines is given in the Appendix. 
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Table 1 


3 ee ee 


Table 2 


RECTANGULAR WINDOW-NF= 25 


WINDOW VALUES 


Wl 4) 
Wt. 2) 
W( «43 
W( 4) 
Wt 3) 
W( 6) 
2 ee & 
W( 8) 
W( 9) 
W( 10) 
W( 11) 
Wi 123 
W( 13) 


- 10000000E 
-10000000E 
- 10000000E 
-10000000E 
-10000000E 
- 10000000E 
- 10000000E 
-10000000E 
- 10000000E 
-10000000E 
- 10000000E 
-10000000E 
- 10000000E 


**LOWPASS FILTER DESIGN+*+* 


IDEAL LOWPASS CUTOFF= 


2 aa fet 
Ht 2) 
Ht = 3) 
mt a 
Ht =-3) 
Bt @) 
| eee 
nt 3 
Bt 9) 
H({ 10) 
H{ 44) 
Hi +2} 
Ht 173) 


PASSBAND CUTOFF 
STOPBAND CUTOFF 


-2807 HZ 
-3176 HZ 


5.2-8 


«39 


BRE 5 
wo 


01 
06 
01 
01 
01 
01 
06 
01 
01 
01 
00 
00 


RIPPLE 
RIPPLE 


ZBZzZALTAZAzS 


. 3000000 
#1359912 70E =0 1 
»27521070E- 
-27820390E- 
- 33636670E- 
-23387470E- 
-26728050E- 
-50455220E- 
-27820390E- 
-75682580E- 
-62366210E- 
-93548720E- 
. 30273090E 
-60000000E 


=f 
=H ( 
=H ( 
=H ( 
=H ( 


.643 DB 
=-79 677 BS 
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Table 3 

HAMMING WINDOW-NF= 30 

ALPHA= .5400000 

WINDOW VALUES 
W( 1)= .80000100E-01 =w( 30) 
W( 2)= .90754870E-01 =wW( 29) 
W( 3)= .12251590E 00 =w( 28) 
W( 4)= .17379830E 00 =w( 27) 
W( 5)= .24220310E 00 =w( 26) 
W( 6)= .32453290E 00 =w( 25) 
W( 7)= .41693830E 00 =w( 24) 
W( 8)= .51509700E 00 =w( 23) 
W( 9)= .61442080E 00 =w( 22) 
W( 10)= .71026440E 00 =w( 21) 
W( 11)= .79814640E 00 =w( 20) 
W( 12)= .87395820E 00 =w( 19) 
W( 13)= .93415440E 00 =w( 18) 
W( 14)= .97592040E 00 =w( 17) 
W( 15)= .99730340E 00 =w( 16) 

**LOWPASS FILTER DESIGN«* 

IDEAL LOWPASS CUTOFF= . 3000000 
H( 1)= .14208050E-02 =H( 30) 


H( 2)= .66123040E-03 =H( 29) 

H( 3)= -.31198420E-02 =H( 28) 

H( 4)= .14865990E-02 =H( 27) 

H( 5)= .59401210E-02 =H( 26) 

H(  6)= -.87972210E-02 =H( 25) 

H( 7)= -.48247580E-02 =H( 24) 

H( 8)= .21861400E-01 =H( 23) 

H( 9)= -.92980680E-02 =H( 22) 

H( 10)= -.33255520E-01 =H( 21) 

H( 11)= .45675080E-01 =H( 20) 

H( 12)= .24561290E-01 =H( 19) 

H( 13)= -.11894020E 00 =H( 18) 

H( 14)= .63996670E-01 =H( 17) 

H( 15)= .51364740E 00 =H( 16) 
PASSBAND CUTOFF .2413 HZ RIPPLE -020 DB 
STOPBAND CUTOFF -3563 HZ RIPPLE -50.398 DB 


Table 4 


TRIANGULAR WINDOW-NF= 25 
WINDOW VALUES 


W( 1)= .76923130E-01 =w( 25) 
W( .2)= -15384620E 00 =W( 24) 
W( 3)= .23076930E 00 =w( 23) 
W( 4)= -107692s04 GU =Wt ° 22) 
W( 5)= .38461540E 00 =w( 21) 
W( 6)= .46153850E 00 =wW( 20) 
W( 7)= .53846160E 00 =w( 19) 
W( 8)= .61538460E 00 =wW(' 18) 
W( 9)= .69230770E 00 =W( 17) 
W( 10)= -76923080E 00 =W( 16) 
W( 11)= .84615390E 00 =w( 15) 
Wt 42)= -92307690E 00 =wW( 14) 
W( 13)= .10000000E 01 =w( 13) 
**LOWPASS FILTER DESIGN«** 
IDEAL LOWPASS CUTOFF= . 3000000 
H ( 1)= -.11993290E-02 =H( 25) 
H( 2)= .42340080E-02 =H( 24) 
H ( 3)= -.64200890E-07 =H( 23) 
H( 4)= -.10349750E-01 =H( 22) 
ee 8 -89951790E-02 =H( 21) 
H( 6)= .12336020E-01 =H( 20) 
H( 7)= -.27168200E-01 =H( 19) 
H( 8)= .17120240E-06 =H( 18) 
H( 9)= £.52395630E-01 =H( 17) 
H( 10)= -.47974010E-01 =H( 16) 
H( 11)= -.79156580E-01 =H( 15) 
H( 12)= .27944390E 00 =H( 14) 
H( 13)= .60000000E 00 =H( 13) 


5.2-9 


FIR Programs 


Table 5 
KAISER WINDOW-NF= 56 
ATT= 60.0000000 BETA= 5.6532570 
WINDOW VALUES 
W( 1)= .20388090E-01 =w( 56) 
W( 2)= .33787730E-01 =w( 55) 
W( 3)= .50466800E-01 =w( 54) 
W( 4)= .70650820E-01 =w( 53) 
W( 5)= .94510080E-01 =w( 52) 
W( 6)= .12215050E 00 =w( 51) 
W( 7)= .15360570E 00 =w( 50) 
W( 8)= .18883060E 00 =w( 49) 
W( 9)= .22769680E 00 =w( 48) 
W( 10)= .26999030E 00 =w( 47) 
W( 11)= .31541080E 00 =w( 46) 
W( 12)= .36357260E 00 =w( 45) 
W( 13)= .41401090E 00 =w( 44) 
W( 14)= .46618500E 00 =w( 43) 
W( 15)= .51949130E 00 =w( 42) 
W( 16)= .57326480E 00 =w( 41) 
W( 17)= .62680230E 00 =w( 40) 
W( 18)= .67936970E 00 =wW( 39) 
W( 19)= .73021390E 00 =w( 38) 
W( 20)= .77858810E 00 =w( 37) 
W( 21)= .82375820E 00 =w( 36) 
W( 22)= .86502400E 00 =w( 35) 
W( 23)= .90173230E 00 =w( 34) 
W( 24)= .93329350E 00 =w( 33) 
W( 25)= .95919230E 00 =w( 32) 
W( 26)= .97900370E 00 =w( 31) 
W( 27)= .99239710E 00 =w( 30) 
W( 28)= .99915160E 00 =wW( 29) 
**BANDPASS FILTER DESIGN** 
IDEAL CUTOFF FREQUENCIES= -1500000 - 3500000 
H( 1)= -.33373340E-03 =H( 56) 
H( 2)= .46434790E-03 =H( 55) 
H( 3)= .27529500E-03 =H( 54) 
H( 4)= .40115490E-03 =H( 53) 
H( 5)= .14646240E-02 =H( 52) 
H( 6)= -.24439120E-02 =H( 51) 
H( 7)= -.26018560E-02 =H( 50) 
H( 8)= .12813520E-02 =H( 49) 
H( 9)= -.16243030E-02 =H( 48) 
H( 10)= .53150390E-02 =H( 47) 
H( 11)= .81133020E-02 =H( 46) 
H( 12)= -.80248120E-02 =H( 45) 
H( 13)= -.37155130E-02 =H( 44) 
H( 14)= -.44724530E-02 =H( 43) 
H( 15)= -.14014040E-01 =H( 42) 
H( 16)= .20645000E-01 =H( 41) 
H( 17)= .19849570E-01 =H( 40) 
H( 18)= -.90004760E-02 =H( 39) 
. H( 19)= .10692340E-01 =H( 38) 
H( 20)= -.33359160E-01 =H( 37) 
H( 21)= -.49442590E-01 =H( 36) 
H( 22)= .48466250E-01 =H( 35) 
H( 23)= .22806540E-01 =H( 34) 
H( 24)= .28850660E-01 =H( 33) 
H( 25)= .99806550E-01 =H( 32) 
H( 26)= -.17628290E 00 =H( 31) 
H( 27)= -.24094420E 00 =H( 30) 
H( 28)= .27797700E 00 =H( 29) 
STOPBAND CUTOFF A171,.8Z.. RIPPLE. .-59..9714 DB 
PASSBAND CUTOFFS -1825 HZ .3174,H2 RIPPLE .012 DB 
STOPBAND CUTOFF -3832 HZ RIPPLE -61.655 DB 
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Table 6 

CHEBYSHEV WINDOW-NF= 55 

DP= .0010000 DF= .0446622 

WINDOW VALUES 
we 1738-4221 746408-01 <wt 55) 
W( 2)= .23350940E-01 =W( 54) 
W( 3)= £.34994060EF-01 =w(-.53) 
W( 4)= .49832560E-01 =w( . 52) 
W( 5S)= .68227770E-01 =w(-*5?f) 
W( 6)= .90503870E-01 =W( 50) 
Wt -7)= . .1169T200E “G6 =W(..493 
W( 8)= .14763970E 00 =W( 48) 
W( 9)= .18275320E -.00 =w( =: 47) 
W( 40)= —.222228505 00: 2W( 2546) 
W( 11)= .26590410E 00 =w(- 45) 
WC 12)= «313504708 °00. =W(-<-44) 
Wl 13)= .364660550E 00 =W( 7548) 
W(-t4)= .41867210E 00 =Ww( -42) 
Wl 145)= <475079S70E 00. aW (2-417) 
W( 16)= . .S53289T00E 00 =W(-40}) 
W( 17)= .591385608 -00-=]W( 4:39) 
W( 18)= .64956160E 00 =W(*°38) 
W{ 19)]=  .70639290E -€0 =¥(-7373 
W( 20)=  .76085360E8 .00 sW(so36) 
W( 21)= .81189510E 00 =W(2: 35) 
W( 22)= .858512008 00 sW(=334) 
W( 23)= .899758005-06 S28 (=-33) 
W( 24)= .93478490E 00 =W( 32) 
W( 25)= .96284500E 00 =Ww(~ 31) 
W( 26)= £.98333280E 00 =W(~ 30) 
w( 27)= ..99580760E ‘00 eW(<s29) 
W( 28)= .10000000E 01 =W( 28) 

**BANDPASS FILTER DESIGN** 

IDEAL CUTOFF FREQUENCIES= .1500000 .3500000 
H( 1)= .10587540E-+-07 =H(.+55) 
H( 2)= .33606800E-03 =H( 54) 
H(t Sye - -F33S7RROE+T3S €Ht 552) 
H( 4)= .77696800E-03 =H( 52) 
H( 5)= -.32665510E-07 =H( 51) 
H( 6)= -.24907520E-02 =H( 50) 
H( 7)= .34595470E-07 #H{ -<49) 
H( 8)= -.15649260E-07 =H( 48) 
H( 9)= .54076300E-07 =H( 47) 
HE 40)= _. 74750520E-02 =H (46) 
H( 11)= -.12730700E-06 =H( 45) 
H( 12)= -.73319790E-02 =H( 44) 
H( 13)= -.45840200E-12 =H( 43) 
H( 14)= -—-.11190420E-01 =H(-. »42) 
H( 15)= .22742580E-06 =H( 41) 
H( 16)= .26887070E-01 =H( 40) 
Ht 17)= .17498860E-06 =H( 39) 
H( 18)= .68851140E-07 =H( 38) 
H( 19)= -.20901980E-06 =H( 37) 
H( 20)= -.57583410E-01 =H( 36) 
H( 21)= .38871140E-06 =H( 35) 
H(.22)=. .S35476708-841 28 34) 
Ht: 2233 377075705 -132. =H(= 33) 
H( 24)= .87448T20E+01 =H (“> 32) 
H( 25)= -.46098210E-06 =H( 31) 
H(. 26)= »297685008. 00. =H(: 30) 
H( 27)= .29465610E-06 =H( 29) 
H( 28)= 240000010E-00 =H(.. 28) 

STOPBAND CUTOFF .1073 HZ RIPPLE -67.961 DB 


PASSBAND CUTOFFS 


STOPBAND CUTOFF 


ADTs He 


.3924 HZ RIPPLE 
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«e062 HZ RIPPLE 


.004 DB 
“67.961 DB 
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NF MUST BE ODD INTEGER FOR HP OR BS FILTERS--NF IS BEING INCREASED BY 1 
CHEBYSHEV WINDOW-NF= 51 


DP= .0010000 DF= .0500000 
WINDOW VALUES 
W( 1)= .16471760E-01 =W( 51) 
W( 2)= .20148630E-01 =W( 50) 
W( 3)= .31728420E-01 =W( 49) 
W( 4)= .46980540E-01 =W( 48) 
W( 5)= .66402670E-01 =W( 47) 
W( 6)= .90445820E-01 =W( 46) 
W( 7)= .11946230E 00 =W( 45) 
W( 8)= .15370090E 00 =W( 44) 
W( 9)= .19324890E 00 =W( 43) 
W( 10)= .23803970E 00 =W( 42) 
W( 11)= .28783570E 00 =W( 41) 
W( 12)= .34218940E 00 =W( 40) 
W( 13)= .40046500E 00 =W( 39) 
W( 14)= .46183130E 00 =W( 38) 
W( 15)= .52529320E 00 =W( 37) 
W( 16)= .58969080E 00 =W( 36) 
W( 17)= .65375500E 00 =w( 35) 
W( 18)= .71612070E 00 =W( 34) 
W( 19)= .77539310E 00 =W( 33) 
W( 20)= .83018530E 00 =W( 32) 
W( 21)= .87916110E 00 =W( 31) 
W( 22)= .92110100E 00 =W( 30) 
W( 23)= .95491870E 00 =W( 29) 
W( 24)= .97973990E 00 =W( 28) 
W( 25)= 99489920E 00 =W( 27) 
W( 26)= 10000000E 01 =W( 26) 


**BANDSTOP FILTER DESIGN«** 


IDEAL CUTOFF FREQUENCIES= 
SF cee Be Sale. 


G 
i) 


+2 9} 
WN OW WWNA uv Ww 


9 
-_ 
= — — — 


PASSBAND CUTOFF 
STOPBAND CUTOFFS 
PASSBAND CUTOFF 


-1500000 . 3500000 
34515350E-13 =H( 51) 
.31414860E-03 =H( 50) 
.15190660E-07 =H( 49) 
.12929490E-02 =H( 48) 
.19648240E-07 =H( 47) 
.95869230E-08 =H( 46) 
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.86167990E-01 =H( 30) 
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.59999990E 00 =H( 26) 
. 1030: HZ’ * RIPPLE .002 DB 
.1974 HZ .3024 HZ RIPPLE > *-73.14/7 DB 
.3968 HZ RIPPLE .002 DB 
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BETA= 5.6532570 
.20388090E-01 =w( 55) 
.34066010E-01 =W( 54) 
.51150610E-01 =W( 533 
.71878610E-01 =W( 523 
.96427080E-01 =w( 51) 
.12490300E 00 =W(_ 50) 
-15733500E 00 =w( 49) 
.19366690E 00 =W( 48) 
(;23375280E -00 =<Bt 47) 
<27735250E 00 =W( 46) 
.322413550E:00 ="¥(. 45) 
.37367830E 00 =wW( 44) 
-42547510E 00 =W( 43+) 
.47893880E 00 =W( 42) 
.53341600E 00 =w( 41) 
.58819190E 00 =W( 40) 
.64251240E 00 =wW( 39) 
.69559720E 00 =Ww( 38) 
.74664920E 00 =w( 37) 
.79488300E 00 =Ww( 36) 
.83953300E 00 =w( 35) 
-87987460E 00 =W( 34) 
.91524270E 00 =W( 33) 
.94504490E 00 =Ww( 32) 
.96877100E 00 =w( 314 
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.99648990E 00 =W( 29) 
.99999900E 00 =W( 28) 

- 3500000 

.74281390E-04 =H( $53 
.24513290E-03 =H( 54) 
.65126990E-03 =H ( 53) 
.56036520E-03 =H( 52) 
-41235770E-03 =H({- 51) 
.17187140E-02 =H( 50) 
.19293910E-02 =H( 49) 
.59010880E-07 =H( 48) 
.31681480E-02 =H( 47) 
.46646410E-02 =H( 46) 
.18755650E-02 =H( 45) 
.43695570E-02 =H( 44) 
.90288630E-02 =H( 43) 
.64007230E-02 =H( 42) 
.40358830E-02 =H( 41) 
.14838590E-01 =H( 40) 
.15041810E-01 =H( 39) 
.21195050E-06 =H( 38) 
.21363840E-01 =H( 37) 
.30079520E-01 =H( 36) 
.11797260E-01 =H( 35) 
.27436900E-01 =H( 34) 
.58266170E-01 =H( 33 
.44204220E-01 =H( a2 
.31763530E-01 =H( 
.14924880E 00 =H( 30) 
.25661440E 00 =H( 29) 
.30000070E 00 =H( 28) 

.3164 HZ RIPPLE -59.842 DB 

.3834 HZ RIPPLE .008 DB 
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Design Subroutine (MXFLAT) for Symmetric FIR Low Pass 
Digital Filters with Maximally-Flat Pass and Stop Bands 


J. F. Kaiser 


Digital Systems Research Dept. 
Bell Laboratories 
Murray Hill, NJ 07974 


1. Purpose 


The program is designed to give the coefficients of a maximally-flat pass and stop band symmetric FIR 
low pass digital filter with an odd total number of terms. The design proceeds from the initial 
specification of the width and the center of the transition band. 3 


2. Method 


In those situations where a low pass (smoothing) filter with a monotonic frequency response charac- 
teristic [1] is necessary the MXFLAT routine gives the coefficients of the desired filter. The MXFLAT 
designs are characterized by a tangency of the filter magnitude-frequency characteristic of L’th order at 
zero frequency and K’th order at half sampling frequency. The combined order is thus 2(K+L-1). 


The frequency response characteristic of a symmetric FIR low pass digital filter having a maximally-flat 
magnitude characteristic is given [2-5] by 


Hy) = ay)" {KaTeaN yp (1) 
where 
y = [1 — cos(2af/f,)]/2 (2) 
and 


K = order of tangency at f = f,/2 
L = order of tangency at f = 0 
f, = sampling frequency in Hz 


Equation (1) is rewritten in a form more convenient for calculation as 
Nr-K (K-~ ‘i 
Hy) = (1-y)K 1+ DY TT + 2h" (3) 
n=] i=] 


where 
N; = K+L-1 = the half-order of the filter. 


To find the actual coefficients of the filter having the magnitude characteristic given by Eq. (1) with the 
substitution Eq. (2) it is necessary to rewrite H(y) in the form 


N. 
H(f) =B(1) + ¥ 2 Bint) cos(2mnf/f,) (4) 


n=] 
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The resulting filter coefficients can then be used directly to filter data; the filtered output data y, is cal- 
culated from the original input data x, as 
Yn = BCI) X, + BQ) Oo, + X12) + BOB) Oxng2 + Xp—2) + ++ * + BONP) (Xpanp-1 + Xp—npa) (5) 


where B(k) is the array of filter coefficients and NP = N \ + Tf. 


FILTER SPECIFICATION DEFINITIONS 


AMPLITUDE 


NORMALIZED FREQUENCY 


Fig. 1 Definitions of 8 and y used in the design of maximally-flat FIR filters. 


Referring to Fig. 1 the specification of N; determines the sharpness of cutoff of the filter as measured 
by y, the normalized width of the transition region defined as the region where the filter magnitude 
response varies from 95% (passband edge) down to 5% (stopband edge). The location of the center, B, 
of the transition region is determined by the ratio K/N;. For these filters all of the K zeros of 
transmission are located at f, /2 rather than being distributed throughout the stopband region as in the 
equiripple FIR designs. 


The design equation relating the width of the transition region to filter order is not known in closed 
form for this filter class. However it can be approximated [1] by 
l 
N-1=2eN, = —~ (6) 
r 2? 

where N—1 is the order of the filter and y is the normalized transition band width as measured by the 
region where the filter response magnitude varies from 95% (passband edge) down to 5% (stopband 
edge). Normalization is with respect to the sampling frequency. These filter parameters are shown in 
Fig. 1. 


Note that in this class of filters the number of terms required is inversely proportional to the square of 
the transition width whereas in the equiripple pass and stop band designs it is inversely proportional to 
only the first power of the transition width. These filter designs thus tend to require more terms for a 
given transition band width. 


3. Program Description 


The package consists of a main program and three subroutines - MKFLAT, PRCOEF, and RATPRX. 
The code for these routines is given in the Appendix. 


After reading in the values of 8 and y, the main program calls up MXFLAT which computes the filter 
coefficients after determining the proper filter order through RATPRX. The filter coefficients are then 
printed out by PRCOEF. The program is dimensioned to handle up to a 398’th order filter. The 
dimension can be increased by changing the value of the variable LIMIT. 


The detailed design proceeds as follows: 


1. Specify 8, the normalized center of the transition band, and y, the normalized width of the transi- 
tion band; the normalization is with respect to the sampling frequency, i.e. 0<B<0.5, 
0<y< minimum (2B, 1-28). If B and y lie outside these limits the design routine prints out a 
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diagnostic to this effect and execution stops. Using the approximate design formula, Eq. (6), a 
lower estimate of the required filter order, N—1, is determined. 


2. The degree of tangency at zero frequency is then determined such as to give the desired location 
for the center, B, of the transition region. The filter order is permitted to change by up to a factor 
of two over that determined in Step 1 in order to adjust the transition band center, B, to as close to 
the prescribed value as possible. 


3. The filter coefficients, B(1), are then determined by first evaluating the frequency response charac- 
teristic at an equally spaced set of frequency points and then performing an inverse discrete Fourier 
transform on this point set. Advantage is taken of the even symmetry in both the frequency 
response and the weighting coefficient sequence. This method of obtaining the coefficients 
simplifies the coding considerably. 


4. Test Problem 


An example of the use of this procedure is shown in Fig. 2. Here beginning with the desired 
specifications of B = 0.2 (center of transition band) and y = 0.1 (transition band width) a filter design 
was produced by MXFLAT with NP=26, N;=25, i.e. a filter with a span of 50 samples. The program 
printed output is displayed in Table 1. The filter coefficients are used according to, Eq. (5). The actual 
resulting B was 0.198 and the y was 0.102 as could be determined from an evaluation of the frequency 
response characteristic. 


B=0.2 
Y=0.4 
NP = 26 
ORDER = 50 


AMPLITUDE 


NORMAL|ZED FREQUENCY 


Fig. 2 Frequency response characteristic of maximally flat (MXFLAT) filter. 


Often in the resulting designs the last few filter coefficients will be virtually zero. This is especially true 

for s mall values of y. These coefficients can be set to zero and the order of the filter reduced accord- 
ingly. The net effect is to distribute the transmission zeros in the upper part of the stopband from their 
clustered position at half the sampling frequency. 


As a general rule the use of this program should be restricted to values of y not much smaller than 
0.05. For example with B=0.2 and y=0.05 the resulting filter had 110 coefficients (218 order), the last 
52 of which were smaller than 0.0000002 in magnitude. 
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Table 1 


Program output for 8B = 0.2, y = 0.1 


FOR BETA = .200 AND GAMMA = .100 
NO. OF COEFS..— 26 
B ( 1) = -39847448 


B ( 2) = -29650429 
B ( 3) = .-08785310 
B ( 4) = —-.05124769 
B ( 5) = —.05604429 
B ( 6) = —.00136329 
B ( 7) = .02472394 
B ( 8) = .01120456 
B ( 9) = —-.00592278 
B( 10) = —.00709112 
B( 11) = —.00061605 
B( 12) = .00232969 
Bt 13} = .00113164 
B( 14) = —.00028640 
B( 15) = —.00043241 
B( 16) = —.00008700 
Bt 17) >= .00006961 
Bt -18) ©= .00004320 
B( 19) = .00000222 
B( 20) = —.00000624 
B( 21) = —.00000241 
B( 22) = —.00000007 
Bt 23) .= .00000016 
B( 24) = .00000002 
B( 25) = —.00000002 
B( 26) = —.00000001 
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A Subroutine for Finite Wordlength FIR Filter Design 


Ulrich Heute 


Institut fuer Nachrichtentechnik 
Universitaet Erlangen-Nuernberg 
Cauerstr. 7 
D-8520 Erlangen, W. Germany 


1. Purpose 


FIR filters in direct form have a transfer function 
np—-l 


fiz= > hw. 


v=0 
with the coefficients h,(v) being identical to the impulse response of the finite length nr. 


Linear-phase filters have a symmetrical impulse response, i.e., 4,(v) = +h,(np—l—v); thus, only 
np +1 
Z ] different elements A,(v), v € {0,1,...,(,—1)} are needed for their description. The 


LS se [ 
frequency response, in this case, becomes purely real or purely imaginary, if a constant delay term is 
neglected. | 


The subroutine IDEFIR /terative Design of Linear-Phase F/R Filters") finds these n, coefficients in a 
way that the frequency response with a rounded fixed-point version, R[h,(v)], of the coefficients 
fulfills a prescribed tolerance scheme. Rounding is carried out to the minimum possible wordlength. 


The tolerance schemes can be of the following types: 


- lowpass filters, 

- highpass filters, 

- symmetrical bandpass filters, 
- symmetrical bandstop filters, 
- HILBERT transformers. 


The tolerance schemes are described by sets of at most six parameters {8), 52,0 p1, 25132, M9}, the 
meaning of which is to be seen from the figures in [3]. All tolerances have to be constant; the sym- 
metry in the bandfilter cases includes identical tolerances in all passbands or all stopbands, respectively, 


and a center frequency at 2 = 0.25 = % =f = f,/4. 


2. Method 
2.1 Principal Idea 
The subroutine IDEFIR realizes a "modified iterative design" procedure as proposed in [1],[2]. 


The main idea is a CHEBYSHEV design of a linear-phase lowpass filter with exact (i.e., computer accu- 
racy) coefficients h,(v) for a transformed and reduced tolerance scheme such that the original require- 
ments are fulfilled with rounded coefficients R[h, (v)]. 


The appropriate reduction is found with the aid of a well-known filter length estimation [3],[4], an 
approximate statistical bound for the effective coefficient rounding error [1],[2],[5], and a minimum 
wordlength estimation derived from this error bound [1]; the "exact design" is carried out by calling any 
convenient lowpass design procedure. Iterations, with modified tolerance reductions, are provided 
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because of the statistical and approximate character of the estimations applied. 
2.2 Design Process 


The design process can easily be understood from the block diagram and the following explanations, 
concerning the numbered stages of the diagram: 


(1) Only the data of the tolerance scheme {81,530 ,1, 0 513;(Q ,2, © ,»)} are needed as input parame- 
ters. 


(2) Filters of the types named in Section 1 can all be derived from equivalent lowpass designs by sim- 
ple transformations (e.g., [4]). 


(3) The minimum wordlength w._ for the (transformed) specifications is estimated. 


(4) The filter length 1; is estimated from the specifications. 


(5) The rounding error effects S,,S, in the passband and the stopband are estimated, taking into 


account the filter length, the wordlength, and the size of "ripples" 5), 5, of the "exact coefficients" 
frequency response. This calculation, itself, is carried out iteratively [1]. 


(6) The estimated total errors, F, = s + $,, F,= 5 + §>, are compared to the prescribed tolerances. 
If the assumption results that the tolerance scheme is fulfilled, a CHEBYSHEV design procedure is 


called. If the tolerances are assumed to be violated, the tolerances a. ~ are reduced; the program 
jumps back to label* 20, and the estimations of n;, S|, and S> are repeated. 


(7) Any convenient design routine can be applied, here. The filter length n,, of the first design itera- 
tion is stored for later comparisons (see point (9)!). 


(8) After coefficient rounding, the real maximum total errors Ejyngx1, Emax. are compared to the 
specifications. If the tolerances are fulfilled, further checks follow (see comment (9a)!). If they 
are violated, a further reduction of 5), 55 is necessary, the program jumps back to label 20, (see 
point (6)!), and the (inner) design iteration loop is re-entered. 

(9) (a) The "exploitation" of the tolerances is checked: if less than a certain percentage p-100 (e.g., 
p = 0.9 = 90%) is used and the filter length exceeds the value n,;, (see comments (9b) and (7)!), 


the tolerances — 8. may be enlarged; in this case, the program jumps back to label 20, as well. 
(b) If no solution was found for the filter length estimated in the first iteration (see point (7)!), 
the length for the first solution found at all takes the place of n,p. 


(10) Tolerance reductions and increases are carried out with different "damping factors" Cr, C;. 
(11) After a change of the design tolerances i 3, it has to be checked, whether the scheme is realiz- 
able, at all. If negative tolerances result, the wordlength is too small. With an increased 


wordlength, the whole (outer) design iteration loop is re-entered, and the program jumps back to 
label 10. 


(12) The (exact and rounded) coefficients of the lowpass fulfilling the requirements are transformed to 
the desired filter type (see comment (2)!). Finally, their "actual wordlength" is determined, i.e., 
the number of bits needed for the sign and for the magnitude of the largest absolute coefficient 
value without leading zeros. 

3. Program Description 

3.1 Main Program 

A main program is needed calling the iterative design routine: 


CALL IDEFIR (DEL1, DEL2, OMGP1, OMGS1, OMGP2, OMGS2, ITYP, NFOO, NF, IWCMIN, 
IWC, IWACT, DEL11, DEL21, EMAX1, EMAX2, HO, RHO, ITERA, IERR). 
The meanings of the parameters in the list are to be found in the "Comment"-part of the subroutine 


* The labels cited here and marked by encircled numbers in the block diagram, correspond to labels in the program listing. 
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Differences between total errors and tolerances: A, , = E 


Subroutine IDEFIR: Iterative DEsign of FIR digital filters (direct form, linear phase) 


Input parameters: 5, 55, Xp, Qs, : 


Minimum coefficient wordlength: we: = w 


Filter length: ne 


Iterative calculation of the expected rounding errors S, 5 
Estimated max. total errors: F, 9: = 8; +S, 2 


Differences between total errors and tolerances: A, , = F; » — 8; > 


<> 


s 


Iteration No. ? 1 


Storage of nr,,,, = Mf = MF 


Design of FIR Chebyshev lowpass with exact 
coefficients for parameters np, 5’; 7, AN 


Rounding of coefficients 


Real max. total errors of the frequency response: E,,,,..\.2 


Check filter type: Lowpass/highpass/bandpass/ bandstop/Hilbert 
Transformation to lowpass design parameters: 6; 7, AN =2,— Ap 


Inner design iteration loop: Design tolerances 5’, 5: = 5, > 


> 


Cmun 


— 81) 


max\,2 


Change of the tolerances: 
Bo: 812 — cry An.2 


Increase of the wordlength: 


Transformation of coefficients to the original filter type. 
Determination of the actual coefficient wordlength 


Output parameters: np, Wc, 5’) >, F; 2, exact and rounded coefficients 


Figure 1 
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7) 


8) 


9) 


10) 


11) 


12) 


FIR Programs Chapter 5 


listing. 


This main program has to provide the input and output of all data; no READ/WRITE - commands are 
included in IDEFIR. 


3.2 Subroutine IDEFIR 


The subroutine receives the design parameters DEL1...OMGS2 defining the tolerance scheme from the 
main program, and it delivers the resulting design parameters DEL1...0MGS1, coefficients, and addi- 
tional information to the calling program (for the single parameters and variables, see COMMENT sec- 
tion of the subroutine listing). The version given in the printout is prepared to handle filters of lengths 
np < 256. A maximum of ITMAX = 20 design iterations is allowed. A p = 90% -tolerance usage is 
aimed at (PROZ = 0.9). Tolerance reductions are carried out by 75% of the tolerance violation value 
(CR = 0.75), increases by 25% of the unused space (CI = 0.25). Changes of p, Cr, C; are possible by 
replacing the corresponding three statements; however, it has to be considered that the choice of these 
values influences the convergence of the algorithm as well as the efficiency of the results: with Cp 
increased and p decreased, fewer iterations may be needed, but the resulting solution may have too 
large a length! On the other hand, larger values C; and p may provide a more efficient use of the toler- 
ances, but they might lead to a program "oscillation"! 


The steps of the subroutine IDEFIR corresponding to the block diagram (Fig. 1) are marked in the list- 
ing in a similar way by comments to be found easily. They are carried out by appropriate subroutines 
to be explained in the sequel. 


3.3 Subroutine TYPTRA 


This routine identifies the filter type from the tolerance scheme by assigning an appropriate value to the 
variable ITYP. Furthermore, it receives the original tolerance scheme from the calling subroutine IDE- 
FIR, and it delivers the tolerances for the equivalent lowpass problem. 


3.4 Subroutine WCMIN 

The minimum possible wordlength JW = We is calculated from the prescribed tolerances D1 = 8), 
D2 = 8), and the transition bandwidth DOMG = AQ = 0. 5;-— 2 pof a lowpass. 

3.5 Subroutine LENGTH 


The filter length NF is calculated from the prescribed tolerances D1 = 6,, D2 = 5,, and the transition 
bandwidth DOMG = AQ = 15-— 2 pof a lowpass. 


3.6 Subroutine EFFERR 


The approximate statistical error bound S is calculated from the tolerance D = 8 of an exact-coefficient 
filter, the length NF = n;, and the coefficient wordlength /W = w, in an iterating loop. The actual 
error formula is evaluated by calling the subroutine SSTAT. 


3.7 Subroutine SSTAT 


The expected error SQ = S/Q (i.e., normalized to the quantization step-size 0 = ee) is calculated 
from the filter length NF = n; and the ratio ALPHA = a = 8/S of original "ripple" size and rounding 
error. 


3.8. Subroutine DESIGN 


Here, any convenient linear-phase, direct-form, FIR lowpass design routine can be applied. Of course, 
the parameter lists in the routine DESIGN and in the CALL DESIGN statement in the subroutine 
IDEFIR have to be adapted. 


In the version given here, DESIGN receives the equivalent lowpass tolerances DEL11 = . : 


DEL21 = “s OMGP1 = 0,,;, OMGS1 = 01,;, and the filter length NF = n,-; it delivers to the calling 
routine IDEFIR the coefficient vector HO = h,(v), v € {0,1,..., (m,;—1)}. 
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3.9 Subroutine MAXERR 


The maximum deviation EMAX = E,,,, of a linear-phase FIR lowpass from the corresponding ideal 
lowpass is calculated. The parameters used are the coefficient vector RHO= R[h,(v)], 

€ {0,1,...,(n;-D}, the length N1 = n, of this vector, the filter length NF = n;, and the frequency 
range [OMGA,OMGE] = [0 4, 0 ¢l, for which the maximum error is wanted. 


The actual error calculation is carried out by calling the subroutine ERROR 
3.10 Subroutine ERROR 


The difference E between an ideal lowpass and a linear-phase FIR lowpass at the frequency point 
OMG = 0 is calculated from the N1 = n, coefficients RHO= R[h,(v)], v € {0,1,...,(n;-1)} of a 
length NF = n, filter. The routine assumes that a fairly close approximation is given: E < 0.5 is 
necessary. 


3.11 Subroutine BACTRA 


According to the information ITYP on the desired filter type, delivered by SUBROUTINE TYPTRA 
(see part 3!), the exact and rounded coefficients, HO = h,(v), RHO= R{h,(v)], of the lowpass filter 
are transformed. The Jowpass filter length NF = n; is coat and the length acs =n, of the desired 
filter is delivered to the calling program. 


3.12 Subroutine WCACT 


The N i m rounded coefficients RHO = R[h,(v)], v € {0,1,...,(m,-1)} are quantized to a stepsize 
Q=2 " °; thus, with a sign bit, WC = w. bits are used to completely define any rounded 
coefficients, in general. 


In special realizations, however, omitting leading zeros behind the binary point might decrease the filter 
wordlength. Therefore, the "actual wordlength" IWACT = w, - (minimum number of leading zeros in 
all coefficients) is determined. 


3.13 DIMENSION and COMMON Requirements 


The routines IDEFIR, DESIGN, MAXERR, ERROR, WCACT contain array specifications for the 
coefficient vectors HO = h,(v) or RHO= R[h,(v)] or both. In the state presented here, a maximum 
of n, = 128 different coefficients is allowed, meaning a maximum filter length of n- = 256. COMMON 
specifications are not contained. If, however, the subroutine DESIGN to be applied needs such 
definitions, they must be inserted into the main program and the subroutine IDEFIR as well. In this 
case, it has to be checked, whether any variables contained in the COMMON definition, are already 
used in subroutine parameter lists. 


4. Summary of User Requirements 


1. Write MAIN program with input/output commands, the subroutine call 

CALL IDEFIR (DEL1, DEL2, OMGP1, OMGS1, OMGP2, OMGS2, ITYP, NFOO, NF, IWCMIN, 
IWC, IWACT, DEL11, DEL21, EMAX1, EMAX2, HO, RHO, ITERA, IERR), 

and the proper DIMENSION and COMMON specifications according to those contained in IDEFIR, 

DESIGN, MAXERR, ERROR, WCACT. 


2. Adapt a convenient DESIGN routine to the subroutine IDEFIR requirements; especially, check 
the parameter lists, DIMENSION, and COMMON specifications. 
5. Test Program and Test Problems 


The routines described above were linked to a main program "IDEFIX" (see the listing below), meeting 
the requirements given in Section 3.1. As a possible design routine, a subroutine-package version of 
the PARKS-McCLELLAN program [6], capable of designing odd-length filters only, was applied. 


Five examples were computed by supplying to the computer the following five data cards: 
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DELI! DEL2 OMGP1 OMGS1 OMGP2 OMGS2 
17. OU! 0.001 0.3 0.35 0.0 0.0 
2) U0) 0.001 0.2 0.15 0.0 0.0 
3). OG 0.001 0.175 0.15 0.325 0.35 
4) 0.01 0.001 0.12 0.15 0.38 0.35 
5) 0.005 0.005 0.025 0.0 0.0 0.0 


The resulting line printer outputs are given below in the tables for the five examples: 


- Lowpass 
- Highpass 
- Bandpass 
- Bandstop 
- HILBERT-Transform 
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Table 1 


ITERATIVE DESIGN OF LINEAR PHASE FIR FILTERS IN DIRECT FORM 
WITH MINIMUM COEFFICIENT WORDLENGTH 


LOWPASS WITH DELTA 1 
| OMEGA P = 
EST. MIN. WORDLENGTH WC MIN = 


0.010000,DELTA 2 0.001000 
0.300000,OMEGA §S 0.350000 
15 BITS (SIGN+BITS BEHIND THE DUAL POINT) 


ESTIMATED LOWPASS LENGTH NF = 55 

COEFFICIENT WORDLENGTH APPLIED WC = 15 

ACTUAL WORDLENGTH WITHOUT LEADING ZEROS WC ACT = 15 
ACTUAL FILTER LENGTH NF = 57 

REDUCED LOWPASS TOLERANCE SCHEME DELTA 1 PRIME = 0.008796 

DELTA 2 PRIME = 0.000593 

EXACT FILTER COEFFICIENTS HO (I), I = 0...((NF-1)/2) 
-0. 0071665 -0.000806 0.003469 0.001039 

~O ..802TS3 0.003388 0.001655 -0 .004896 

0.003558 0.003980 -0.008106 0.002993 

0.008183 “SU TT TTS 0.000822 0.014755 
-0.015467 -0.004170 0.024699 =0 ,.018993 

~0 024353 0.040839 -0.021880 -0.037062 

0.075071 =@. 02378) -“Uy Leese 0.286443 

0.642227 

ROUNDED FILTER COEFFICIENTS HO (1) 

-0.001831 -0 .000793 0.003418 0.001038 
-0.002136 0.003357 0.001648 -0.004883 

0.003540 0.003967 -0.008057 0.002997 
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Table 1 
(Continued) 

0.008179 -0.011719 0.000793 0.014709 
-0.015442 -0.004150 0.024658 -0.018982 
-0.014343 0.040833 -0.021851 -0.037048 

0.075012 -0.023743 -0.122925 0.286438 

0.642212 

ACTUAL TOLERANCES OF THE EQUIVALENT LOWPASS WITH ROUNDED COEFFICIENTS 

DELTA 1 = 0.008044,DELTA 2 = 0.000671 

NUMBER OF DESIGN ITERATIONS = 4 

Table 2 


ITERATIVE DESIGN OF LINEAR PHASE FIR FILTERS IN DIRECT FORM 
WITH MINIMUM COEFFICIENT WORDLENGTH 


0.001000 
0.150000 


HIGHPASS WITH DELTA 1 
OMEGA P 


0.010000,DELTA 2 
0.200000,OMEGA §S 


FWFIR 


EQUIVALENT LOWPASS WITH DELTA 1 = 0.010000,DELTA 2 = 0.001000 
OMEGA P = 0.300000,OMEGA §S = 0.350000 
EST. MIN. WORDLENGTH WC MIN = 15 BITS (SIGN+BITS BEHIND THE DUAL POINT) 


ESTIMATED LOWPASS LENGTH NF = _ 55 


COEFFICIENT WORDLENGTH APPLIED wC = 15 

ACTUAL WORDLENGTH WITHOUT LEADING ZEROS WC ACT = 15 
ACTUAL FILTER LENGTH NF = 57 

REDUCED LOWPASS TOLERANCE SCHEME DELTA 1 PRIME = 0.008796 

DELTA 2 PRIME = 0.000593 

EXACT FILTER COEFFICIENTS HO (I), I = 0...((NF-1)/2) 
-0.001865 0.000806 0.003469 -0.001039 
=0.0Q2783 -0.003388 0.001655 0.004896 
0.003558 -0.003980 -0.008106 -0.002993 
0.008183 0.011719 0.000822 -0.014755 
-0.015467 0.004170 0.024699 0.018991 
-0.014353 -0.040839 -0.021880 0.037062 

0. 875071 o. 02378) -0 “T22956 -0.286443 
0.642227 

ROUNDED FILTER COEFFICIENTS HO (TI) 

~0 001831 0.000793 0.003418 -0.001038 
-0.002136 -0.003357 0.001648 0.004883 
0.003540 -~0 .003967 -0.008057 -0.002993 
0.008179 0.073719 0.000793 -0.014709 
-0.015442 0.004150 0.024658 0.018982 
-0.014343 -0.040833 oO. O02180 1... 0.037048 
0.075042 0.023743 “0.422925 -0.286438 
0.642212 


ACTUAL TOLERANCES OF THE EQUIVALENT LOWPASS WITH ROUNDED COEFFICIENTS 


DELTA 1 = 0.008044,DELTA 2 = 0.000671 
NUMBER OF DESIGN ITERATIONS = 4 
Table 3 


ITERATIVE DESIGN OF LINEAR PHASE FIR FILTERS IN DIRECT FORM 
WITH MINIMUM COEFFICIENT WORDLENGTH 


BANDPASS WITH DELTA 1 = 0.010000,DELTA 2 = 0.001000 

OMEGAP1 = 0.175000,OMEGAS1 = 0.150000 

OMEGAP2 = 0.325000,OMEGAS2 = 0.350000 
EQUIVALENT LOWPASS WITH DELTA 1 = 0.010000,DELTA 2 = 0.001000 

OMEGA P = 0.150000,OMEGA S = 0.200000 

EST. MIN. WORDLENGTH WC MIN = 15 BITS (SIGN+BITS BEHIND THE DUAL POINT) 
ESTIMATED LOWPASS LENGTH NF = 55 
COEFFICIENT WORDLENGTH APPLIED we = 15 


ACTUAL WORDLENGTH WITHOUT LEADING ZEROS WC ACT = 14 

ACTUAL FILTER LENGTH NF = 117 

REDUCED LOWPASS TOLERANCE SCHEME DELTA 1 PRIME 0.008362 
0.000430 


DELTA 2 PRIME = 
) 


EXACT FILTER COEFFICIENTS HO (I), I = 0...((NF-1)/2 
0.000607 0, -05 000572 as 
-0.000520 QO. 0.002465 0. 
=O, 003505 i8 0.901655 a; 
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Table 3 
(Continued) 

0.001906 0. -0.004329 03 

0.002120 ne 0.003647 0. 
-0.007141 03 0.003230 0. 

0.005906 6:3 -0.010919 0. 

0.004360 o 0.009573 0. 
-0.016369 6. 0.005519 0. 

0.015509 a: -0.024539 0. 

0.006580 0. 0.025888 0. 
-0.038408 0. 0.007432 0. 

0.047958 0. -0.070128 0. 

0.007984 0. 0.132416 0. 
-0.279054 0. 0.341510 

ROUNDED FILTER COEFFICIENTS HO (TI) 

0.000549 0. -0.000549 0. 
-0.000488 0. 0.002441 a: 
-0.003479 0. 0.001831 Gs, 

0.001892 0. -0.004272 0. 

0.002075 0. 0.003601 0. 
-0.007141 og 0.003174 0. 

0.005859 0. -0.010864 O., 

0.004333 0. 0.009521 O% 
-0.016357 0. 0.005493 0. 

0.015503 o. -0.024536 0. 

0.006531 0. 0.025879 0. 
-0.038391 e: 0.007385 0. 

0.047913 0; -0.070068 i 

0.007935 62 0.132385 0. 
-0.279053 0. 0.341492 

ACTUAL TOLERANCES OF THE EQUIVALENT LOWPASS WITH ROUNDED COEFFICIENTS 

DELTA 1 = 0.009576,DELTA 2 = 0.000987 

NUMBER OF DESIGN ITERATIONS = 12 

Table 4 


ITERATIVE DESIGN OF LINEAR PHASE FIR FILTERS IN DIRECT FORM 
WITH MINIMUM COEFFICIENT WORDLENGTH 


BANDSTOP WITH DELTA 1 


OMEGAP 1 
OMEGAP2 


EQUIVALENT LOWPASS WITH 


EST. MIN. WORDLENGTH WC 
ESTIMATED LOWPASS LENGTH NF 


COEFFICIENT WORDLENGTH APPLIED 


0.010000,DELTA 2 
0.120000, OMEGAS1 
0.380000 ,OMEGAS2 
DELTA 1 = 
OMEGA P = 
T2 “Bits 


MIN 


47 


WC 


0.001000 
0.150000 
0.350000 
0.010000,DELTA 2 
0.240000,OMEGA S 


0.001000 
0.300000 


(SIGN+BITS BEHIND THE DUAL POINT) 


= 15 


ACTUAL WORDLENGTH WITHOUT LEADING ZEROS 
ACTUAL FILTER LENGTH 
REDUCED’ LOWPASS. TOLERANCE SCHEME 


EXACT FILTER COEFFICIENTS HO0- (13)..-T = 0... 
-0.001520 O -0.003752 
-0.000894 a. 0.003676 

0.000481 0. -0.005578 

0.000340 O. 0.008082 
-0.001983 a. -0.011180 

0.005010 0. 0.014701 
-0.010025 a. -0.018449 

0.017968 0. 0.022139 
-0.030583 0. -0.025456 

0.052234 Oe 0.028100 
-0.098901 0. -0.029799 

0.315849 0: 0.530385 

ROUNDED FILTER COEFFICIENTS HO (TI) 
-0.001465 5 -0.003723 
-0.000854 0. 0.003662 

0.000427 0. -0.005554 

0.000305 0. 0.008057 
-0.001953 Oo; -0.011169 

0.005005 a 0.014648 


NF 


95 
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WC ACT 


DELTA 1 PRIME 
DELTA 2 PRIME 


OOo) ©: ere 


Co O'S: OS OO OS Oi OO) Oi-s 


(NF=1-) 72 


) 


Ta 


0.009881 
0.000676 
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Table 4 
(Continued) 
-0.010010 0. -0.018433 0. 
0.017944 0. 0.022095 0. 
-0.030579 0. -0.025452 0. 
0.052185 0. 0.028076 0. 
-0.098877 0. -0.029785 0. 
0.315796 0. 0.530334 
ACTUAL TOLERANCES OF THE EQUIVALENT LOWPASS WITH ROUNDED COEFFICIENTS 
DELTA 1 = 0.009237,DELTA 2 = 0.000885 
NUMBER OF DESIGN ITERATIONS = 1 
Table 5 


ITERATIVE DESIGN OF LINEAR PHASE FIR FILTERS IN DIRECT FORM 
WITH MINIMUM COEFFICIENT WORDLENGTH 


HILBERT-TRANSFORMER WITH DELTA = 0.005000,OMEGAP = 0.025000 
EQUIVALENT LOWPASS WITH DELTA 1 = 0.002500,DELTA 2 = 0.002500 
OMEGA P = 0.225000,OMEGA S =.0.275000 
EST. MIN. WORDLENGTH WC MIN = 14 BITS (SIGN+BITS BEHIND THE DUAL POINT) 


ESTIMATED LOWPASS LENGTH NF = 59 


COEFFICIENT WORDLENGTH APPLIED wc = 14 


ACTUAL WORDLENGTH WITHOUT LEADING ZEROS WC ACT = 14 

ACTUAL FILTER LENGTH NF = 59 

REDUCED LOWPASS TOLERANCE SCHEME DELTA 1 PRIME = 0.002137 
DELTA 2 PRIME = 0.002137 

EXACT FILTER COEFFICIENTS 30-(1), I= G.. at (NF*1)72) 

0.002960 0.000055 0.003471 0.000055 

0.005015 0.000055 0.007564 0.000055 

0.010351 0.000055 0.014415 0.000055 

0.019086 0.000055 0 5025550 0.000055 

0.033402 0.000055 0.044355 0.000055 

0.059084 0.000055 0.081828 0.000055 

0.120482 0.000055 0. 206221 0.000055 

0.635134 0. 

ROUNDED FILTER COEFFICIENTS HO (TI) 

0.002930 0. 0.003418 O.. 

0.004883 oy 0.007324 Og 

0.010254 0. 0.014404 O-. 

0.019043 0. 0. 025391 0: 

0.033203 Os 0.044189 Oe. 

0.059082 2 0.081787 0. 

0.120361 = 0.208008 0: 

0.635010 G. 


ACTUAL TOLERANCES OF THE EQUIVALENT LOWPASS 
DELTA 1 = 0.002485,DELTA 2 = 0.002422 
NUMBER OF DESIGN ITERATIONS = 1 


WITH ROUNDED COEFFICIENTS 
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CHAPTER 6 


IIR Filter Design and Synthesis 


J. F. Kaiserand K. Steiglitz 


The problems of designing infinite impulse response (IIR) and finite impulse response (FIR) digital 
filters differ in a number of significant ways. First, in the important special cases of lowpass, highpass, 
and bandpass magnitude specifications, we have available in the IIR case the classical closed-form 
designs associated with the names of Butterworth (maximally-flat pass and stop bands), Chebyshev 
(equiripple pass or stop band), and Cauer (elliptic, equiripple pass and stop bands) originally developed 
for the continuous case. 


The first program in this chapter, DOREDI by G. F. Dehner, is devoted to the design, implementation, 
and analysis of finite wordlength IIR digital filters based on such closed form designs. This very 
comprehensive program includes the capability of designing the classic lowpass, highpass, bandpass, and 
bandstop filters from specifications on cutoff frequencies and passband and stopband tolerances. The 
program has the facility to determine from the extended degree infinite-precision coefficient design the 
minimum coefficient wordlength assuming coefficient rounding necessary to meet the filter 
specifications. There is also the facility to optimize the noise performance of the filter by changing the 
pairing and ordering of the poles and zeros of the filter being realized as a cascade of second-order sec- 
tions. These last two powerful features can also be used directly on filter designs input separately. Thus 
the Dehner program provides a very comprehensive IIR filter design capability. 


When the design of an IIR digital filter with arbitrary magnitude or phase characteristics is needed, we 
encounter another important difference between the IIR and FIR cases. The general IIR design problem 
is highly nonlinear in the filter coefficients, whereas the FIR problem is linear. This means that general 
purpose design programs are necessarily more complicated and time consuming than their FIR counter- 
parts. Two such programs are included here: LPIIR by A. Deczky, based on the minimum p-error cri- 
terion [1], and OPTIIR by M. T. Dolan and J. F. Kaiser, which uses the penalty function method. 
LPIIR minimizes the p' power of the error, and so includes the least-square-error case (when p=2) 
and the minimax criterion (when poo). Typically p=40 is sufficient for the minimax criterion. LPIIR 
also provides an option for phase equalization. 


Two different general purpose IIR design programs are included here because these optimization prob- 
lems can be quite difficult numerically. Any one given program may fail to converge on some problem, 
especially one of high order. Also, we have no mathematical guarantee that the solutions obtained are 
globally optimal, since the convergence criterion is usually that the gradient of the error be very small 
in norm - a necessary but not sufficient condition for global optimality in general nonlinear design prob- 
lems. The reader may therefore wish to try a given problem on both programs, and compare the solu- 
tions if they both converge within the allotted execution time. Even in the cases where convergence is 
obtained and the specifications are satisfied, different computers may give different results. These 
differences can be attributed to different computer word lengths, different rounding rules, different 
conversion routines, and different FORTRAN compilers. 


The last program, FWIIR by K. Steiglitz and B. D. Ladendorf, deals with the problem of choosing finite 
word-length coefficients in a cascade implementation of an IIR filter. The program requires the user to 
provide an "infinite-precision" design in this form. This problem is doubly difficult, since it is not only 
nonlinear in its parameters, but its parameter space is discrete. No algorithm is known which is practical 
for problems with as many as 12-16 parameters and which also guarantees optimality, so it is necessary 
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to use a heuristic. FWIIR uses a randomized version of the Hooke-and-Jeeves search algorithm [2,3], 
which attempts to build a pattern of successful moves on the grid in parameter space. The fact that it is 
randomized means that different runs may produce different local optima; the user can then select. the 
best. 


FWIIR yields results which are very close to those reported by Avenhaus and Schuessler [4], who also 
use a search on a discrete grid. Recently Brglez [5] presented a method which improves on some pub- 
lished examples by 1 or 2 bits. He does a sequence of discrete local optimizations interspersed with con- 
tinuous optimizations, thus allowing excursions in parameter space larger than would be obtained by 
search on a discrete grid only. Brglez’s paper appeared too late for inclusion of his program here. The 
reader is also referred to related work on the IIR problem [6,7,8] and the FIR problem [9,10]. 
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6.1 


Program for the Design of Recursive Digital Filters 


G. F. Dehner 


Institut fiir Nachrichtentechnik 
Univ. Erlangen-Nurnberg, Cauerstrasse 7 
D-8520 Erlangen, Germany 


1. Purpose and Method 


The general purpose program, DOREDI, an acronym for Design and Optimization of REcursive Dlgital 
filters, is designed to solve the classic approximation problem for recursive digital filters, to minimize 
the coefficient wordlength and then to minimize the noise performance by scaling and pairing and ord- 
ering the poles and zeros for a number of different structures, and finally to analyze the performance of 
the resulting filter. 


A general view of the design possibilities of DOREDI is given in the block diagram in Fig. 1. 


The analysis and noise optimization procedures not only can be applied to filters generated within the 
design program, but also can be used for the investigation of other filter designs by inputting separately 
their filter coefficients. 


1.1 Design of selective [1R-filters 


The filter types that can be handled are the lowpass, highpass, symmetrical bandpass and bandstop 
filters. The types of approximation available are the maximally-flat magnitude (Butterworth), the Che- 
byshev types I and II, and the elliptic (Cauer) forms. The method used is the transformation of the 
tolerance scheme into the continuous domain, the approximation, and then the transformation back to 
the discrete domain [1], [2]. 


1.2 Optimization of the coefficient wordlength 


The optimization is obtained by choosing an appropriate filter degree and, thus, a certain margin inside 
the given tolerance scheme (necessary extension of the filter degree, see [3],[4]). This margin, unused 
for filters with a high coefficient wordlength, is distributed between passband and stopband such that 
the required wordlength of the rounded coefficients is small (trial and error procedure) [3], [4]. 


1.3 Optimization of the state variable wordlength 


For cascade structures with given second-order blocks, the output noise and consequently the 
wordlength of the state variables [1] is optimized by pairing poles and zeros and ordering the subsys- 
tems; a heuristic method is used called "optimization with limited storage" and is described in [4]. For 
a system of lower order, this procedure is equivalent to the "dynamic programming" approach [5] and 
leads to an exact solution for systems including independent white noise sources, only. 


In order to reduce storage and computing time only the S,,,,, best intermediate results are stored and 
handled in the consecutive step. For S,,,, = 1, the procedure equals the "direct procedure" explained in 
[4] and [6]. For general noise sources (see Appendix B) the optimization procedure considers the 
correlations within the second-order blocks, only. 


1.4 Analysis of the optimized filter 


Input: Coefficients designed by the program or specified independently with input data cards (class 
01, 03, 04, and 05, see sect. 2.2). 
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Output: Optimized coefficients, 
required additional wordlength Aw 
Analysis of the designed filter 


Fig. 1 Block diagram of DOREDI. 


(a) Test of the necessary coefficient wordlength. 
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(b) Calculations of the output noise and the additional wordlength of the state variables; all correla- 


tions of the noise sources within the total filter are taken into account. 


(c) Printer plots of the magnitude and the phase of the frequency response as well as the impulse 


response. 
1.5 Realization of the transfer function 


In the cascade realization, the transfer function is given by 


H( ) Il ee ss 512 + bo, 
Zz = ee 
| z*>+ 4,2 > Coy 
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The coefficient and state variable wordlengths can be optimized for different hardware realizations. 
1.5.1 Coefficients 
The options available are: 


(a) fixed-point sign-magnitude realization 
(b) "pseudo"-floating-point realization 
(c) realization by differences 


For a detailed description of the different realizations see Appendix A and [4]. 
1.5.2 State Variables 


Fixed-point two’s complement representation: The necessary reduction of the wordlength inside the 
filter can be done by different arithmetics and at different points within the filter structures. For a 
detailed description see Appendix B and [4], [6], [7], [8]. | 


1.5.3 Structures of Second-Order Blocks 


The structures available are variations of the first and second canonic forms (Fig. 2) with or without 
separate scaling factors. 


First Form Second Form 


Fig. 2 Signal-flow graph of the canonic forms. 


Scaling can be realized by a multiplication with an arbitrary value or by a power of two [4], [6], [10]. 


2. How to Use DOREDI - Description of the Input Data 


The whole design and optimization program is controlled by the main program calling the on-line input 
data compiler and the proper program sections sequentially for the different tasks. All input data are 
read in via the on-line input compiler, allowing an almost free ordering of the data cards within one 
program section. 


2.1 Call of the program sections and the input compiler 

Normal program section card format 

"//" marks the program section card. "name" means the program section 
name; only the first two characters are significant. A program section is 


/ /name closed by another program section card or by a finish card with the charac- 
ter string: //Flnish. 


= 2 ee 


Program section format without DECODE 


Normally the input data card is processed by the "DECODE" statement. If "DECODE" does not con- 
form to the compiler in use, an alternative program section input can be installed by compilation (see 
sect. 4.1). The original program section card is split up into a special control card "//" and a second 
card with the program section name beginning in column 3. 
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Description of the program sections 
//SYnthesis: Design of recursive digital filters with or without optimized coefficients 


//SEquence: Optimization of the pairing and ordering of second-order blocks achieving 
minimum output noise 


//SS (Synthesis-Sequence) 
This task includes the sections //SY and //SE. For a realization of second- 
order blocks without separate scaling factor, another optimization of the 
coefficient wordlength is done in a third step taking into account the sequence 
of the second-order blocks fixed in the step before (Fig. 1). 


//ANalysis: Analysis of the digital filter designed before or defined by input data cards. 


2.2 Control parameters and filter data input 
Normal input formats 


Two formats are used for the input of the design parameters (tolerance scheme, program control 
parameters, filter realization, etc.): 


(a) The first one is the control card with a unified format for one LOGICAL, one INTEGER and 
three REAL parameters: 


FORMAT(1A1,212,2X,L1,1X,15,1X,3E15.0,12A1) 
12345 8 10-14 16-30 31—4 5 6 6-60 


) bielethidmeaae if i$  @——ie=esa 
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In the following, the different input variables are described by a short notation: 
*IIJJ LINTEG,REAL,REAL,REAL; COMMENT 


If any data arrays within one set of data are superfluous, they are marked by "bb', following 
superfluous arrays are not marked. 


Description of the data arrays 


*IJJ: Input code 
II, Input class 
JJ, Number within a class 
Input cards to be used alternatively belong to the same decade (e.g., 
JJ=41 or JJ=42). 


L LOGICAL input 
FORMAT: LI1 T = TRUE 
F, any other character, or blank = FALSE 
INTEG INTEGER input 
FORMAT: IS, right-justified 
REAL REAL input 


FORMAT: unformatted floating-point read statement within the specified array, 
if necessary with a right-justified exponent x.xxE+ee 


(b) For the input of INTEGER arrays (e.g., a predetermined pairing and ordering, or the exponents 
of the "pseudo"-floating point coefficients) a second type of format, the INTEGER data card, is 
necessary. This data card can only be used as a continuation card of a control card. 


FORMAT (1A1,212,3X, 1614) 
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Scaling can be realized by a multiplication with an arbitrary value or by a power of two [4], [6], [10]. 


2. How to Use DOREDI - Description of the Input Data 
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program section. 
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ter string: //FInish. 
Program section format without DECODE 


Normally the input data card is processed by the "DECODE" statement. If "DECODE" does not con- 
form to the compiler in use, an alternative program section input can be installed by compilation (see 
sect. 4.1). The original program section card is split up into a special control card "//" and a second 
card with the program section name beginning in column 3. 
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Description of the program sections 
//SYnthesis: Design of recursive digital filters with or without optimized coefficients 


//SEquence: Optimization of the pairing and ordering of second-order blocks achieving 
minimum output noise 


//SS (Synthesis-Sequence) 
This task includes the sections //SY and //SE. For a realization of second- 
order blocks without separate scaling factor, another optimization of the 
coefficient wordlength is done in a third step taking into account the sequence 
of the second-order blocks fixed in the step before (Fig. 1). 


//ANalysis: Analysis of the digital filter designed before or defined by input data cards. 


2.2 Control parameters and filter data input 
Normal input formats 


Two formats are used for the input of the design parameters (tolerance scheme, program control 
parameters, filter realization, etc.): 


(a) The first one is the control card with a unified format for one LOGICAL, one INTEGER and 
three REAL parameters: 


FORMAT(1A1,212,2X,L1,1X,15,1X,3E15.0,12A1) 
12345 8 10-14 16-30 31—4 5 6 6-60 
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In the following, the different input variables are described by a short notation: 
*IIJJ LINTEG,REAL,REAL,REAL; COMMENT 


If any data arrays within one set of data are superfluous, they are marked by "bb", following 
superfluous arrays are not marked. 


Description of the data arrays 


*TJJ: Input code 
II, Input class 
JJ, Number within a class 
Input cards to be used alternatively belong to the same decade (e.g., 
JJ=41 or JJ=42). 


L LOGICAL input 
FORMAT: LI T = TRUE 
F, any other character, or blank = FALSE 
INTEG INTEGER input 
FORMAT: IS, right-justified 
REAL REAL input 


FORMAT: unformatted floating-point read statement within the specified array, 
if necessary with a right-justified exponent x.xxE+ee 


(b) For the input of INTEGER arrays (e.g., a predetermined pairing and ordering, or the exponents 
of the "pseudo"-floating point coefficients) a second type of format, the INTEGER data card, is 
necessary. This data card can only be used as a continuation card of a control card. 


FORMAT (1A1,212,3X, 1614) 
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INO2 INO2 INO3 IN45 IN(46 


Description of the data arrays 


*00JJ Input code for the INTEGER data card. II has to be 00 because this format is 
only used for data input following a control card. JJ Number of the leading 
control card 


INO1...IN16 INTEGER-array 
FORMAT: 1614, right-justified 


Input formats without "DECODE' 
2 ee 


According to sect. 2.1 a second data input format can be installed. 
I I J J eeoe = Ry : won 
i. = ££ For this purpose, all data cards are split up into a control card and 


a oe a second card with input code and data in the normal input format. 


Description of the input parameters 


In the following, the different input data are explained. They are classified according 
to their functions: 


Class 00 - data input (only used in connection with a control card) 
Class 01 - general input/output control 

Class 02 - parameters for the optimizations 

Class 03 - tolerance scheme 

Class 04 - realization of the coefficients 

Class 05 - realization of the blocks of second order 

Class 06 - special analysis of the designed filter. 


The use of some of the input data classes makes no sense, if called in an individual program 
section. Therefore, Table 1 shows (x) the principally allowed input data classes for the 
different program sections. 


As far as it is useful, default values are assigned to the input parameters; in addition to the 
definition of the tolerance scheme, the input is reduced to those data (or data cards, respec- 
tively) disagreeing with the default values. 


At the beginning of the program all parameters agree with their default values. When a new 
program section within a program run is reopened, only the parameters within the input data 
classes, marked by "d" in Table 1, are redefined by their default values. 


01 x X X Xx 


Table 1 Allowed input data classes (x) and redefinition of the default values (d). 
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2.2.1 General Input/Output Control — Data Cards of Class 01 
2.2.1.1 Input of filter coefficients (for the program sections SE and AN) 


*0110 
NINP 
NINP=1: 


NINP=?: 
NINP=3: 


NINP=4: 


Default value: 


bb, NINP 
Input parameter 


Unrounded coefficients and, if not explicitly specified by input data, the realiza- 
tion parameters from the previous program section 


See NINP=1, but from a disk file (channel number KA4) 


Input of second-order blocks from the card reader; the following data cards are 
expected to immediately follow the control card): 

*0010 bb, NB, FACT 

*0010 bb, bb, B2 (01),B1(01),B0(01) 


*0010 bb,bb,B2 (NB) ,B1 (NB), BO (NB) 
*0010 bb, bb,bb,Cl (01),C0(01) 


*0010 bb,bb,bb,C1 (NB),CO(NB) 


Input of the zeros and poles; the following data cards are expected immediately 
behind the control card *): 

*0010 bb, NB,FACT 

*0010 bb, bb, ZZR (01) ,ZZ1(01) 


*0010 bb,bb,ZZR (NB),ZZI(NB) 
*0010 bb,bb,ZPR (01),ZPI(01) 


*0010 bb;bb,ZPR (NB) ,ZPI(NB) 
only the zeros and poles in the closed upper complex plane have to be declared. 


NINP=1 


2.2.1.2 Output of the results on the line printer 


*0120 
NOUT 
NOUT=0: 


to 


NOUT=5: 


bb, NOUT 
INTEGER parameter for the output on the line printer (channel number KA2) 


No line printer output 


Complete line printer output with all intermediate results 


For the intermediate results, see output example in section 3. 


Default value: 


NOUT=3 


2.2.1:3 Output of the results on the disk 


*0130 

LDOUT 
True: 
False: 


LDOUT 

LOGICAL parameter for the output on the disk (channel number KA4) 
An output is provided according to the ASCII format given below 

No disk output 


+ For the meaning of the names see section 2.2.1.3. 
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Default value: LDOUT=.FALSE. 
List of formatted records for the output on the disk 


> === DOREDI === VERSION V005 ===’ 


’FILTER DESCRIPTION’ 
ITYP,IAPRO,NDEG 
SF,OM(1),...,0M(4),ADELP, ADELS 
AC,ROM(1),...,ROM (4), RDELP,RDELS 
NZM(1),...,NZM(4),M 
ZM(1,1),...,2M(1,4) 


ZM(M,1),....ZM(M,4) 

‘DATA’ 

NB 

FACT 
B2(1),B1(1),BO(1),C1(1),CO(1) 


B2(NB),B1 (NB), BO(NB),C1 (NB),CO(NB) 
IRCO(1),...,.IRCO(5) 
IECO(1,1),...,[ECO(1,5) 


IECO(NB, 1),....IECO(NB,5) 
IDCO(1,1),...,.IDCO(,5) 


IDCO(NB, 1),...,IDCO(NB,5) 
IWL,IECOM, JMAXV,ITRB2 
"END OF DATA’ 


All INTEGER records in the format 10I5 
All REAL records in the format 4E15.7 


Description of the parameters 


ITYP = filter type 

[IAPRO = kind of approximation 

NDEG = filter degree 

SF = sampling frequency 

OM (1) = cutoff frequencies in radians 
ADELP = ripple in the passband (s) 

ADELS = ripple in the stopband(s) 

AC = design parameter C 

ROM (I) = changed cutoff frequencies 

RDELP = changed ripple in the passband (s) 
RDELS = changed ripple in the stopband(s) 
NZM (I) = number of the different extrema 

M = max {NZM(J)}; J = 1,2,3,4 

ZM (I, 1) = location of the maxima in the passband 
ZM (I, 2) = location of the minima in the passband 
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ZM (1,3) 
ZM (I, 4) 
NB 
FACT 


Chapter 6 


= location of the maxima in the stopband 
= location of the zeros in the stopband 
= number of blocks of second order 


= gain factor 


B2(1),B1(1),BO(1) = numerator coefficients 


C1(1),co() 
ZZR (1) 

ZZ1 (1) 

ZPR (I) 

ZPI (1) 


denominator coefficients 


real part of the zeros in the z-plane 


imaginary part of the zeros in the z-plane (ZZI (I) =0, real zero) 


real part of the poles in the z-plane 


= imaginary part of the poles in the z-plane (ZPI(I) =0, real pole) 


IRCO(N),IFCO(L,N),IDCO(L,N), IWL,IECOM,JMAXV and JTRB2 are described in sect. 


2.2.4. 


2.2.1.4 Special output 


*0140 
LSPOUT 


True: 


False: 
ISPOUT: 


Default values: 


LSPOUT,ISPOUT 
LOGICAL parameter for the special output 


Output is provided; a special output procedure "OUTSPE" has to be defined and 
linked to the program by the user (see sect. 4). 


No special output 
Free parameter for the special output 


LSPOUT=.FALSE. 
ISPOUT=0 


2.2.2 Parameters for the Optimization — Data Cards of Class 02 


2.2.2.1 Optimization of the coefficient wordlength 


*0210 
LWLF 


True: 


False: 
ITERM 
ACX 
ACXMI 
ACXMA 


Default values: 


*0220 


LWLF,ITERM,ACX,ACXMI,ACXMA 


LOGICAL parameter for the optimization of the tolerance scheme utilization 
with a fixed wordlength 


Optimization of the parameters ACX for a fixed coefficient wordlength IWL; 
IWL can be defined by the control card *0410 (See sect. 2.2.4.1) 


No fixed wordlength; coefficient wordlength will be minimized 

Maximum number of optimization cycles for the final coefficient optimization 
See section 2.2.3.2 (for the alternative input format see section 4.1) 

Lower bound for the design parameter ACX: 0. < ACXMI < 1. 

Upper bound of the design parameter ACX: 0. < ACXMA < 1. 


LWLF=.FALSE. 

ITERM=10 

ACX, see *0320 in section 2.2.3.2 and Appendix D 
ACXMI=0. 

ACXMA $1. 


bb, ITERM1 
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ITERM1 


Default value: 


EQUR 


Maximum number of optimization cycles for the pre-optimization (see sect. 
2.1, //SS) 


ITERM1=4 


Optimization of the output noise 


*0230 


LPAIRF 


True: 


False: 


ISTOR 


Default values: 


*0240 
LSEQ 


True: 


False: 
ISCAL 
ISCAL=1: 


ISCAL=2: 


ISCAL=3: 


SCALM 


Default values: 


LPAIRF,ISTOR 


The program section call //SE or //SS starts the output noise optimization 
automatically. This control card is only necessary to alter the default values. 


LOGICAL parameter for a fixed pairing 


Optimization of the ordering of blocks of second order with a fixed pairing of 
the poles and zeros, according to a rule of thumb [1], [4], [6], or to the input 
by the control card *0240. For structures 21, 23, 25, and 27 the input *0240 
provides fixed pairing of the auxiliary blocks (Appendix C). ATTENTION: 
fixed pairing is not possible for the structures 22, 24, 26, and 28. 


Free optimization of the pairing and ordering 


Maximum number of intermediate storage S,,,, for the "optimization with lim- 
ited storage" [4], ISTOR < 100 


LPAIRF=.FALSE. 
ISTOR = 100 


LSEQ,ISCAL,SCALM 


LOGICAL parameter for the input of a fixed pairing and ordering (e.g., as a 
start ordering for the heuristic optimization) 


Input of a fixed pairing of the zeros and poles. Two input data cards have to 
follow immediately: 

*0040 a INTEGER data format 

*0040 ISEQD (1),...,ISEQD (16) 

The numerator and denominator are numbered in an ascending order. 
ISEQN(I) =J means the J-th numerator out of the input sequence is taken as 
the I-th numerator in the final realization. The same is true for the denomina- 
tor declaration ISEQD (I). 


No input 
INTEGER parameter for the kind of scaling [1], [4] 


absolute criterion 
Dd |fo(k)| < SCALM + 
k=0 


criterion for sinusoid input 
L.. = max|F;(Q.)| < SCALM f¢ ¢ 


criterion for the power of a white noise source 


L3=+ f lf)? dO < SCALM¢ 3 
0 


7 
chosen maximum of the overflow points 


LSEQ=.FALSE. 
ISCAL=2 
SCALM=1. 


+ The criterion has to be satisfied for all overflow points within the realized structure. 


+ L,=L,norm; L., = L, norm; F, see Appendix B. 
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2.2.3 Description of the Desired Filter — Data Cards of Class 03 
2.2.3.1 Type of the filter; sampling frequency 


Ra ee es 


Pee Ee 


*0310 
ITYP 
ITYP=1: 
ITYP=2: 
ITYP=3: 
ITYP=4: 
SF 


Default values: 


bb, ITYP,SF 

Filter type 

lowpass filter 

highpass filter 

symmetrical bandpass filter 
symmetrical bandstop filter 


Sampling frequency (input may as well be executed with the data cards *0341 
or *0342) 


ITYP=1 
SF no default value provided 


Type of approximation; design parameter 


*0320 

IAPRO 
IAPRO=1: 
IAPRO=2?: 
IAPRO=3: 
IAPRO=4: 

ACX 


ACX=0.:: 
to 
ACX=1.: 


Default values: 


bb, [APRO, ACX 

Type of approximation 
Butterworth (maximally flat) 
Chebyshev I (passband) 
Chebyshev II (stopband) 
Elliptic filter (Cauer) 


Normalized design parameter C for the utilization of the tolerances in the 
passband and the stopband [3],[4] (start value for the optimization) 
0.<ACX<1. (See Appendix D) 


the passband tolerance is used completely 


the stopband tolerance is used completely 


IAPRO=4 (elliptic filter) 

ACX, such that passband and stopband are used as equally as possible. (For 
the alternative input format see section 4.1; in this case for the default input, 
ACX has to be set to ACX=1000.) 


Filter degree (order of the recursive system) 


*0330 
LDEGF 


True: 


False: 
NDEG 
EDEG 


Default values: 


LDEGF,NDEG,EDEG 


LOGICAL parameter related to the specification of a filter degree. For the 
check of the filter order, the degree extension [4] is introduced into the calcula- 
tion. 


If the tolerance scheme cannot be fulfilled with the prescribed degree, the pro- 
gram run is terminated. 


If the given degree is not sufficient, the order is increased automatically. 
Prescribed filter order 
Relative extension of the degree [4]. 


LDEGF=.FALSE. 
NDEG see remarks 
EDEG=0.2 (for the alternative input format see section 4.1) 


6.1-10 


Section 6.1 EQUR 


Remarks: 


a) Normally, the degree is determined automatically. The minimum degree 
N min fOund in this way is not necessarily an integer value. Taking this 
into account as well as including the degree extension for the purpose of a 
minimized coefficient expenditure [3],[4], yields the order to be used as 


An 
[n dat - | ‘on ] rounded 
A min 
n= MA 
<n min> half—adjusted 
AF a = EDEG 
min 


b) maximum filter degree is set to Nmax = 32 


c) To obtain the minimum filter degree for a prescribed tolerance scheme, 
the degree extension EDEG has to be set explicitly to EDEG=0.0. 


2.2.3.4 Cutoff frequencies for the passband(s) and stopband(s) 


The cutoff frequencies are read in by one (and only one) of the following sets of data: 


fsa fips for fs2 fayo — f 
DO oq py Npz Msp T —S 22 
O Ysy Ypy Vp2 Ys2 » —— : 
Fig. 3. Cutoff frequencies (e.g., for a bandpass). 
(a) *0341 bb, bb, FR (1) ,FR(2),<SF> 
and for bandpass and bandstop filters in addition 
*0342 bb, bb, FR (3), FR (4),<SF> 
FR (1) Unnormalized cutoff frequencies 


FRO) 


FR (4) 


SF Sampling frequency; superfluous, if already defined in data card *0310. The 
units of FR(I) and SF have to be identical - i.e., both Hz or kHz. (<> means 
that this parameter may be omitted.) 


(b) *0343 bb,bb,OM(1),0M (2) 


and additionally for bandpass or bandstop filters 
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(c) 


(d) 


*0344 
OM (I) 
*0345 


*0346 
S (1) 


*0347 
VSN 


VD 
A 


Remarks: 
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bb,bb,OM (3),0M (4) 

Cutoff frequencies in radians, normalized by the sampling frequency 
bb,bb,S(1),S(2) 

and additionally for bandpass and bandstop 

bb, bb,S(3),S(4) 


Cutoff frequencies of the corresponding analog system (input if analog filters 
are to be designed) 


bb,bb, VSN,VD,A 


Normalized stopband cutoff frequency v,’ of the corresponding analog lowpass 
filter (actual design parameter for the transition from passband to stopband [1]) 


Passband edge frequency of the corresponding analog lowpass filter, VD = vp 


Parameter for the transformations from the normalized lowpass to the bandpass 
and bandstop filters. 


a) 


b) 


Prescription of only three cutoff frequencies for symmetrical band filters. 
If four frequency values are prescribed, the requirements for the calcula- 
tion of the reference lowpass are overdetermined. Without further data 
input, the tolerance scheme is utilized such that a minimal filter degree is 
achieved (see section 2.2.3.6). If only three frequency values are given, 
the fourth one is automatically chosen in a manner yielding an analog 
band filter which is symmetric with respect to its center frequency. Exam- 
ple: 


0 ¥s1 Vp Vs2 
Vp4 Vp2 
Fig. 4. Free choice of the cutoff frequency Vp» for a symmetric bandpass. 


Determination of the transition band. The filter order and the tolerances 
5p and 8, being given, the transition bandwidth of the filter can be deter- 
mined automatically. A required degree extension for the optimization of 
the coefficient wordlength is taken into account. Lowpass example: 


ee | 
| ; 
n=6 . n=6 
ee 
schceidtindadieabubbad eh SU 535 ee at te Peres 


Fig. 5. Free choice of one cutoff frequency for a lowpass. 


For a band filter, remark a) can be regarded in addition; then, only two 
cutoff frequencies have to be prescribed. Bandpass example: 
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Fig. 6. Free choice of two cutoff frequencies for a bandpass. 
2.2.3.5 Tolerances in passband and stopband 
The passband and stopband tolerances are read in by one of the following data cards, optionally: 
oO bb,bb, ADELP, ADELS 


ADELP Tolerance 6p in the passband 
ADELS Tolerance 6, in the stopband 
(b) *0352 bb,bb, AP, AS 
AP Tolerated passband attenuation ap (in dB) 
AS Stopband attenuation ax (in dB) 
(c) *0353 bb,bb,P,AS 
r Reflection coefficient p 
P=p=/1- (1-8,)? 
Remarks: The filter order, the transition band, and one of the 6 tolerances being given, 


the other one can be determined. A required degree extension for the optimi- 
zation of the coefficient wordlength can also be regarded. Examples are given 
in Fig. 7. 


Lowpass | Highpass 


Bandpass 


Fig. 7. Free choice of one of the tolerances. 


2.2.3.6 ‘Choice’ of the symmetrical band filter 


*0360 bb, NORMA 
NORMA Parameter for the determination of the reference lowpass of symmetrical 
bandfilters 
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NORMA =0: calculation of vp such that 7,,;, is minimized; 
NORMA =1: calculation of vp from the inner cutoff frequencies; 
NORMA =?: calculation of vp from the outer cutoff frequencies; 


NORMA#=3: calculation of vp as the geometric mean of all cutoff frequencies. Examples 
for a bandpass filter are shown in Fig. 8. 


Default value: NORMA=0 


satel 


/\IW 


Vs4 Vey Vp Vp2 
Vs2 


NORMA = i: Vp = ~/ Vpi1Yp2 NORMA = Fa Vp = V/Vs1V52 


VSN = 1.18 VSN = 1.12 
here, this is equivalent to NORMA = 0 


Oe 
NORMA =3 
[\/VN 


Vs2 my Ie Vp? 
Ys? 


VSN = 1.14 


<< (vsyvpiv pars.) 


Fig. 8. Different choice of the parameter NORMA 
2.2.3.7 Output of the analog filter 


*0370 LSOUT 

LSOUT LOGICAL parameter for the special output 
True: Output of poles, zeros, and 2nd order blocks of the corresponding analog filter 
False: No special output 


Default value: LSOUT=.FALSE. 
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2.2.4 Realization of the Coefficients — Data Cards of Class 04 


General coefficient representation (see Appendix A) 
ACO(L,N) =ACOS(L,N)*2.** (IRCO(N) — IECO(L,N)) 


— SIGN (2.**IDCO(L,N),ACOS(L,N)) 


Description of the parameters 


2.2.4.1] 


2.2.4.2 


ACO(L,1) = B2(L) = 55, 
ACO(L,2) = BI(L) = 6, 
ACO(L,3) = BO(L) = bo, 


ACO(L,4) = C1(L) 
ACO(L,5) = CO(L) 


Cin 
COy 


ACOS(L,N) = sign + mantissa of the "pseudo"-floating-point coefficients 
IRCO(N) = exponent of the number range 


IECO(L,N) = 


"pseudo"-floating-point exponent 


IDCO(L,N) = exponent of the difference branch 


Coefficient wordlength 


*0410 
LSTAB 


True: 


False: 


IWL 


Default value: 


Attention: 


LSTAB,IWL 
LOGICAL parameter for the stability test 


Check only whether the system with rounded coefficients is stable; no correc- 
tion. 


Check whether the system with rounded coefficients is stable and change, if 
necessary. 


Wordlength of sign and mantissa; quantization Q, = 2>¢W'-!) 
IWLMI < IWL < IWLMA; for IWLMI and IWLMA see sect. 4. 


LSTAB=.FALSE. 
IWL = 16 


IWL is only used for the optimization of the coefficient errors. For other 
inputs apply IWLR in *0530. 


Number range of the coefficients 


*0420 
RCOINP 


True: 


False: 
JRCO 
JRCO<0 
JRCO>0 
JRCO|=1 
JRCO|=2 


RCOINP,JRCO 


LOGICAL parameter for the input of a predetermined array of coefficient 
ranges 


One additional INTEGER-data card is necessary: 
*0020 IRCO(1),...,IRCO(S) 


No extra input. 

Parameter for the calculation of the necessary coefficient ranges 
ranges IRCO(I) <0 allowed 

ranges IRCO(I) [0 only 

separate ranges for B2(L),B1(L),BO(L),C1(L),CO(L) 


one common range for (B2(L),B1(L),BO(L)); separate range for C1(L) and for 
CO(L) 
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JRCO|=3 
JRCO|=4 
JRCO|=5 


Default values: 


Chapter 6 


one common range each for (B2(L),B1(L),BO(L)) and for (C1(L),CO(L)) 


common ranges for (B1(L),C1(L)) and (BO(L),CO(L)) 


one common range for all coefficients 


RCOINP =.FALSE. 
JRCO=2 


2.2.4.3 ’Pseudo’-floating-point coefficients 


*0430 
ECOINP 


True: 


False: 
JECO 

JECO=0 

JECO+0 


Default values: 


ECOINP,JECO 

LOGICAL parameter for the input of predetermined exponents 

Five additional INTEGER-data cards are necessary: 

*0030 IECO(1,1),...,IECO(NB, 1) 

*0030 IECO(1,5),...,IECO(NB,5) 

NB = number of second-order blocks used 

No extra input 

Parameter for the calculation of the exponents 

no "pseudo"-floating-point realization 

value of the "pseudo"-floating-point exponent must be less or equal to JECO 


ECOINP=.FALSE. 
JECO=0 


2.2.4.4 Realization by differences 


*0440 
DCOINP 


True: 


False: 
JDCO 

JDCO=0: 

JDCO+0: 


DCOINP,JDCO 
LOGICAL parameter for the input of predetermined difference branches 
Five additional INTEGER-data cards are necessary: 


*0040 IDCO(1,1),...,.IDCO(NB, 1) 


*0040 IDCO(1,5),...,IDCO(NB,5) 
NB = number of second-order blocks used 
IDCO(L,N) = —100 means no difference branch for this coefficient 


No extra input 
Parameter for the calculation of the difference branches 
no realization with differences 


realization with differences is possible. JDCO is used as a code word; the digits 
are assigned to the control parameters JJDCO(N) of the different coefficient 
classes: 


JDCO = JIDCO(S) for CO(L) 
JIDCO(4) for C1(L) 
JJDCO(3) for BO(L) 
JJDCO(2) for B1(L) 
JIDCO(1) for B2(L) 


JIDCO(N) = 1: common difference branch realization for all blocks 
(block multiplexing) 


JIDCO(N) =2: separate realization; only for those blocks with 
coefficients less than half the original number range 
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Default values: DCOINP=.FALSE 
JDCO=00000 


2.2.4.5 Realization of the largest numbers within the range 


*0450 bb, JMAXV 

JMAXV Parameter for the control of the largest numbers 
JMAXV=1: -28 Cx <2 
JMAXV =2: —2R cx < 2’ two’s complement 
JMAXV=-2: -—2% < x < 2 and 2'® — 2'®—LSB representation 
JMAXV=3: =78 ag <7 sign-magnitude 
JMAXV=-3 —2" < x < 2'® and —2' = —2'°+LSB representation 

2° — 2'®_-LSB 


LSB = Least significant bit 


Remark: JMAXV=—2,-—3 is useful to reduce the necessary number range 

Default values: JMAXV=—3 

*0460 bb, JTRB2 

JTRB2 Parameter to control the rounding of the scaling factors; correction of the scal- 
ing error 


JTRB2<0: normal rounding of the coefficients with an influence on scaling; violation of 
the scaling is possible 


JTRB2>0: truncation of these coefficients 
JTRB2|=1 only the total gain factor FACT is corrected 


JTRB2|=2 the scaling of the next block is corrected (useful for structures with separate 
scaling factors (see sect. 2.2.5.1)) 


JTRB2|=3: in addition to JTRB2=2, the coefficients B1(L) and BO(L) of the same block 
are corrected (useful for structures without separate scaling factors) 


Default values: JTRB2=3 
2.2.5 Realization of the Cascade Structures with Blocks of Second Order — Data Cards of Class 05 


The implemented structures for second-order blocks with five or four (zeros on the unit circle) 
coefficients are shown in Figs. 9 and 10. The nonlinearities are marked by a square box with an N over 
it. The characteristics of these nonlinearities are declared by a control card. For scaling, because of the 
two’s complement realization of the state variables, only the overflow points marked by — have to be 
checked. 


2.2.5.1 Structures of second-order blocks 


*0510 LPOT2,ISTRU 

LPOT2 LOGICAL parameter for the choice of the scaling factor 
True: Scaling by powers of two (bit shifts) 
False: Scaling with any factor 
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STRUCTURE 17 STRUCTURE 18 


Fig. 9. Blocks of second order in the first canonic form. 
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STRUCTURE 27 STRUCTURE 28 


Fig. 10 Blocks of second order in the second canonic form. The coefficient 5; is only 
realized in the first block of a cascade; for the calculation of the noise, the 
part, separated by L___~ is attached to the following block; b, = b/b 


* only realized for the last block in the cascade 
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ISTRU 


Default value: 


Chapter 6 
Number of the structure of the second-order blocks to be realized (see catalog 
in Figs. 9 and 10). 


LPOT2=.FALSE. 
ISTRU=13 


2.2.5.2 Characteristics of the nonlinearities 


*0520 
CNOINP 


True: 


False: 
JCNO 


Default value: 


*0530 
LCNO 


True: 
False: 


IWLR 


Default value: 


CNOINP,JCNO 


LOGICAL parameter for the input of inhomogeneous nonlinearity characteris- 
tics for the different second-order blocks 


Five additional input data cards are necessary to characterize the nonlinearities 
*0020 ICNO(1,1),....ICNO(NB, 1) 


*0020 ICNO(1,5),...,ICNO(NB,5) 
NB = number of second-order blocks 


ICNO(L,N) = control parameter according to JJICNO (see next section); in 
this case JCNO is unaffected 


No extra input 


parameter for the input of the nonlinearity characteristics, unified for all blocks 
JCNO is used as a code word; the digits are assigned to the five nonlinearities, 
according to JJCNO 


ICNO = 


Zeros are replaced by the values of the nearest succeeding nonzero digit, e.g., 
00205 = 22255. The different characteristics are described by the parameter 
JJCNO (see Appendix B). 


JJICNO=1: two’s complement rounding 

JJCNO=2: sign magnitude rounding 

JICNO=3: mathematical rounding 

JICNO=4: two’s complement truncation 

JICNO=5: unsymmetrical sign magnitude truncation 
JICNO=6: symmetrical sign magnitude truncation 
JIJICNO=7: random rounding 


Especially for small coefficients, these different characteristics are distinct in 
their generation of random noise, DC offset, and errors correlated with the sig- 
nal [4]. For the error values see Appendix B. 


CNOINP =.FALSE. 
JCNO = 33333 


LCNO,IWLR 


LOGICAL parameter for the consideration of the coefficient quantization by 
the calculation of the output noise (see Appendix B). 


No coefficient quantization is considered 
Noise parameters will be adjusted (see Appendix B, Table B-1). 


parameter for the specification of the realized coefficient wordlength. In the 
noise analysis, the wordlength may be different from that found by the 
coefficient optimization. 


LCNO=.FALSE. 
IWLR=100, (no coefficient rounding) or result of the coefficients optimization 
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2.2.6 Analysis of the Designed Filter — Data Cards of Class 06 


bb04 


2dsOek 


siesce 


2.2.6.4 


2.202 


Coefficient analysis 
*0610 LWLM,IWL 
LWLM LOGICAL parameter for the search of minimum wordlength | 
True: For a given set of coefficients and realization the necessary coefficient 
wordlength will be found. 
False: Error analysis is done for a fixed wordlength IWL. 
IWL Coefficient wordlength 
Default value: LWLM=.FALSE. 
IWL=16 


Noise analysis 

For a given realization, the output noise and the necessary additional wordlength Aw are calcu- 
lated. 

*0620 LSEQ,ISCAL,SCALM 


For description of the parameters see control card *0240, sect. 2.2.2.2. In addi- 
tion if ISCAL=—1 no scaling is done. 


Printer plot of the magnitude of the frequency response 
*0630 LNOR,IPAG,OMLO,OMUP,RMAX 


Response in linear scale 


LNOR LOGICAL parameter for the normalization 
True: Normalization to the extreme values in the given range 
False: No normalization 
IPAG Number of line printer pages to be used for one plot 
OMLO Lower limit for the abscissa (normalized frequency in radians) 
OMUP Upper limit for the abscissa (normalized frequency in radians) 
RMAX Maximum value for the ordinate 
Default values: LNOR=.FALSE. (range from 0. to RMAX) 
IPAG =2 
OMLO=0. (for the alternative input format see sect. 4.1) 
OMUP=7 
RMAX = 1. 
Printer plot of the attenuation of the frequency response 


*0640 LNOR,IPAG,OMLO,OMUP,RMAX 
Response in dB, for parameter description see sect. 2.2.6.3. 


Default values: LNOR=.FALSE. (range from 0. to RMAX) 
IPAG=2 
OMLO=0.; (for the alternative input format see sect. 4.1) 
OMUP=7 
RMAX=100 dB 


Printer plot of the phase response 
*0650 LNOR,IPAG,OMLO,OMUP 
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For parameter description see sect. 2.2.6.3. 
Default values: LNOR=.FALSE. (range from —7z to +77) 


IPAG=2 
OMLO=0.; (for the alternative input format see sect. 4.1) 
OMUP=7 
2.2.6.6 Printer plot of the impulse response 
*0660 LNOR,IPAG,RMIN,RMAX 
LNOR Parameter for the normalization, see sect. 2.2.6.3. 
IPAG Number of line printer pages 
RMIN Minimum value for the ordinate of the unnormalized plot range 
RMAX Maximum value for the ordinate of the unnormalized plot range 
Default values: LNOR=.FALSE. (range from RMIN to RMAX) 
IPAG=2 
RMIN=-1. 
RMAX=1. 


3. Examples for the Filter Design and Optimization 
3.1 Design of a filter without optimization 


3.1.1 Specifications 


Lowpass: 
sampling frequency ft, = 200 kHz 
cutoff frequency in the passband fp = 2.0 kHz 
cutoff frequency in the stopband (/; = 4.0 kHz 
passband tolerance dp = 0.02 
Stopband tolerance 55 = 0.001 
3.1.2 Execution of the Design 


extension of the filter degree Boy 0.2 


min 
equal use of passband and stopband 
3.1.3 Input Data Cards 


77 SY 

*0310 au. 

*0341 _s 4. 
*0351 0.02 0.001 
7#7FEX 


3.1.4 Output on the Line Printer 
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3.1.4.1 Normal output, according to the default output code NOUT = 3 


=== DOREDI === VERSION vV005 === 
DATA INPUT 
//SY 
* ee 0 0.200000E 02 C. 0. 
* 3 41 0 0.200000E 01 0.400000E 01 0. 
* : ae 0 0.200000E-01 0.100000E-02 O* 
/7FY 
TOLERANCE SCHEME 
FILTER-TYPE 
LOWPASS 
APPROXIMATION 
ELLIPTIC 
SAMPLING FREQ. 20.000000 
CUTOFF FREQUENCIES 2.000000 4.000000 
NORM. CUTOFF FREQ. 0.628319 1.29663? 
CUTOFF FREQ. S-DOM. 0.324920 0.726543 
PASSBAND RIPPLE(S) 0.020000 0.1755 DB P = 0.1990 
STOPBAND RIPPLE(S) 0.001000 60.0000 DB 
MIN. FILTER DEGREE 4.6285 
DEGREE EXTENSION 0.9257 
CHOSEN FILTER DEG. 6 
BOUND PAIR OF THE DESIGN PARAMETER C S200) ik 2 UE: 61.6686 
REALIZED 
NORM. CUTOFF FREQ. 0.626319 1.256637 
CHOSEN DESIGN PAR. CX = 0.6632134 (=) ¢ = 2269879777 
UTILIZATION OF THE PASSBAND DELTA P = 0.0028525 = 14.26252 PERCENT 
STOPBAND DELTA S = 0.0001432 = 14.32376 PERCENT 
POLES AND ZEROS IN THE Z-DOMAIN 
CONSTANT GAIN FACTOR = 0.1559554E-02 
NUM. POLES NUM. ZEROS 
2 O. 747153. -+-J% 0.590872 2 0.280642 +-J* 0.999812 
2 0.675963 +-J« 0.412069 2 0.000000 +-Jx 1.000000 
2 0.666428 +-Jx 0.1 Ser 1 . -0.753474 +-J 0.657478 
BLOCKS OF SECOND ORDER 
CONSTANT GAIN FACTOR = 0.1559554E-02 
L B2(L) B1 (L) BO(L) C1 (L) CO(L) 
1 1.00000000 -0.56128475 1.00000000 -1.43430551 0.86343788 
2 1.00000000 -0.00000001 1.00000000 =|, o0toe0'6 0.62672679 
3 1.00000000 1.50694796 1.00000000 -1.33285540 0.46668934 
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3.1.4.2. Complete line printer output, according to the greatest output, code NOUT = 5 


=== DOREDI === VERSION V005 === 
DATA INPUT 
eg > 4 
* : eee 5 i? 0. 0. 
* a 10 0 0.200000E 02 .. o 
* 3 41 0 0.200000E 01 0.400000E 01 0. 
* Sie 0 0.200000E-01 0.100000E-02 0% 
//¥FI 
TOLERANCE SCHEME 
FILTER-TYPs 
LOWPASS 
APPROXIMATION 
ELLIPTIC 
SAMPLING FREQ. 20.000000 
CUTOFF FREQUENCIES 2.000000 4.000000 
NORM. CUTOFF FREQ. 0.628319 1.256637 
CUTOFF FREQ. S-DOM. 0.324920 0.726543 
PASSBAND RIPPLE(S) 0.020000 0.4735 25 P= 0.1990 
STOPBAND RIPPLE(S) 0.001000 60.0000 DB 
NORMALIZED PARAMETER IN THE S-DOMAIN 
VD = 0.324920 
VSN oa 2.236068 
MIN. FILTER DEGREE 4.6285 
DEGREE EXTENSION O.S2o7 
CHOSEN FILTER DEG. 6 
CAP. DELTA 0.003293 
ZEROS OF THE CHARACTERISTIC FUNCTION /K(J*V)/**2 
2 0. 9093S99 .+-J* 0. 
2 0.726543 +-J* Us 
2 O. 2/2822 *=9% 0; 
BOUND PAIR OF THE DESIGN PARAMETER C Shee! ates Ae ee 61.6686 


POLES AND ZEROS OF THE NORMALIZED REFERENCE LOWPASS IN THE S-DOMAIN 


NUM. POLES NUM. ZEROS 
2 0. +-J* 2.306654 
2 0. +-J* 3.077684 
2 O. +-J* 8.208093 
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NOUT 


IV 
ON 


ul 
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POLES AND ZEROS OF THE REFERENCE FILTER IN THE S-DOMAIN 


NUM. 


REALIZED 


NORM. CUTOFF FREQ. 


CHOSEN DESIGN PAR. CX = 


UTILIZATION OF THE PASSBAND DELTA P 
STOPBAND DELTA S 


POLES 


0.628319 


0.66321 


NUM. ZEROS 

2 +-J* 0.749477 

2 +-J* 1.000000 

2 +-J* 2.666971 
1.256637 

24. {nj = 22.9879777 

= 0.0028525 = 14.26252 PERCENT 
= 0.0001432 = 14.32376 PERCENT 


POLES AND ZEROS OF THE NORMALIZED REFERENCE LOWPASS IN THE S-DOMAIN 


CONSTANT GAIN FACTOR = 


NUM. 


2 
2 
é 


POLES AND ZEROS OF THE 


POLES 


-0.127449 +-Jx 
-0.385683 +-J* 
-0.586296 +-J* 


CONSTANT GAIN FACTOR = 


NUM. 


y 
2 
2 


POLES AND ZEROS IN THE 


POLES 


-0.041411 +-J* 
-0. 125316 +-J# 
=O. 190499. 4+-J+ 


CONSTANT GAIN FACTOR = 


NUM. 


2 
2 
2 


POLES 


0.7 17133-.4-0% 
0.6/5963 +-J* 
0.666428 +-Jx 


BLOCKS OF SECOND ORDER 


CONSTANT GAIN FACTOR = 


Wn 


B2(L) 


-00000000 
-00000000 
-00000000 


B1 (L) 


-0.56128475 1 
-0.00000001 1 
1.50694796 1 


0.143237 


1.102886 
0.851538 
0.330270 


0.143237 


0.358349 | 


0.276681 
G.107371 


Z-DOMAIN 


0.155955 


0.590872 
0.412069 
0.150271 


0.155955 


6E-03 


NUM. 


6E-03 


NUM. 


4E-02 


NUM. 


4E-02 


BO(L) 


-00000000 
-00000000 
-00000000 


ooo 


~4 
1 
~1 
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REFERENCE FILTER IN THE S-DOMAIN 


ZEROS 
+-J* 
+-J* 
+-J* 


ZEROS 
0.280642 +-Jx 
0.000000 +-J 

-0.753474 +-Jx 


C1(L) 


-43430551 
- JOTS2Z616 
» 33285540 


0.749477 
1.000000 
2.666971 


0.959812 
1.000000 
0.657478 


CO(L) 
0.86343788 
0.62672679 
0.46668934 
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EXTREMES OF THE MAGNITUDE OF THE TRANSFER FUNCTION (COEFS NOT ROUNDED) 


MAXIMA IN THE PASSBAND (P) 
MAXIMA IN THE STOPBAND (S) 


BAND S-DOMAIN Z-DOMAIN 

IN RAD IN DEGREE 
P 0.0885 0.1766 10.1168 
S 0.7265 1.2566 72.0000 
P 0.52361 0.4636 26.5635 
S 0.8277 1.3826 79.2305 
P 0.3150 0.6103 34.9664 
S) 1.3948 +<8975 106.7275 
S0.4254E 38 3.1416 180.0000 


MINIMA IN THE PASSBAND (P) 
MINIMA IN THE STOPBAND (S) 


BAND S-DOMAIN Z-DOMAIN 

IN RAD IN DEGREE 
? is 0. 0. 
Ss 0.7495 tx aeos 43% TOTS 
P 0.1693 C.. 3255 79 2128 
S 1.0000 1.5708 90.0000 
. 6 +2852 0.5957 31.8366 
S 2.6670 2.4241 138.8922 
e 0.3249 0.6283 36.0000 


3.2 Optimized design of a lowpass filter 
3.2.1 Specifications (see sect. 3.1.1) 
3.2.2 Chosen Realization 


oo -o- Oo = 


MAGNITUDE 


.000000 
.000143 
.000000 
.000143 
.000000 
.000143 
.000143 


MAGNITUDE 


OOO" So Oe 


.997147 
-997147 
-000000 
.997147 


.997147 
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Structure 16; first canonic form with separate scaling factors; mathematical rounding after each multi- 


plier; scaling by powers of two. 


3.2.3 Necessary Input Cards 


J /7SS 

*0210 20: G2 

*0230 T 

*0310 aus 

*0341 ins 4. 
*0351 6.. 02 0.001 
*0420 1 

*0450 1 

*0510 T 16 

//¥FI 


3.2.4 Line Printer Output of the Designed Filter 


=== DOREDI === VERSION V005 === 
DATA INPUT 
//88 
* es 3,” 20 0.200000E 00 0. 
* * ee | 2 eee 0 0. Os 
* eee & 0 0.200000E 02 0 
* 3 41 0 0.200000E 01 0 
* coe 0 0.200000E-01 0 
* 4 20 1 0. 0. 
* 4 50 1 D::. 0. 
* erie) eae 16 0. 0. 
//¥FI 


-400000E 01 
-100000E-02 
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TOLERANCE SCHEME 


FILTER-TYPE 


LOWPASS 
APPROXIMATION 
ELLIPTIC 
SAMPLING FREQ. 20.000000 
CUTOFF FREQUENCIES 2.000000 4.000000 
NORM. CUTOFF FREQ. 0.628319 1.256637 
CUTOFF FREQ. S-DOM. 0.324920 0.726543 
PASSBAND RIPPLE(S) 0.020000 0.1755 DB P = 0.1999 
STOPBAND RIPPLE(S) 0.001000 60.0000 DB 
MIN. FILTER DEGREE 4.6285 
DEGREE EXTENSION 0.9257 
CHOSEN FILTER DEG. 6 
BOUND PAIR OF THE DESIGN PARAMETER C 3229272 .LEs Oke; 61.6686 
REALIZED 
NORM. CUTOFF FREQ. 0.628319 1.296637 


OPTIMIZATION IS TERMINATED AFTER THE 10. STEP 
THREE UNSUCCESSFUL STEPS 


CHOSEN DESIGN PAR. Cx = ©.0031250 (=) C= 3.3230284 
UTILIZATION OF THE PASSBAND DELTA P = 0.0000599 = 0.29930 PERCENT 
STOPBAND DELTA S-7 = 0.0009909 = 99.08854 PERCENT 
POLES AND ZEROS IN THE Z-DOMAIN 
CONSTANT GAIN FACTOR = 0.6750014E-02 
NUM. POLES NUM. ZEROS 
2 0.621103 +-J* 0.646824 Z 0.280642 +-Jx 0.959812 
2 0.532684 +-J* 0.440233 Z 0.000000 +-Jx 1.000000 
2 0.481727 +-Jx 0.159623 2 -0.753474 +-Jx« 0.657478 
BLOCKS OF SECOND ORDER 
CONSTANT GAIN FACTOR = 0.6750014E-02 
L B2(L) B1(L) BO(L) C49 CO(L) 
1 1.00000000 -0.56128475 1.00000000 -1.24220650 0.80415063 
2 1.00000000 -0.00000001 1.00000000 -1.06536840 0, 47755731 
2 1.00000000 1.50694796 1.00000000 -0.96345360 0.25754030 
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EXTREMES OF THE MAGNITUDE OF THE TRANSFER FUNCTION (COEFS NOT ROUNDED) 


MAXIMA IN THE PASSBAND (P) 
MAXIMA IN THE STOPBAND (S) 


BAND S-DOMAIN Z-DOMAIN MAGNITUDE 
IN RAD IN DEGREE 
P 0.0885 0.1766 10.1168 1.000000 
S 0.7265 1.2566 72.0000 0.000991 
P 0.2361 0.4636 26.5651 1.000000 
S 0.8277 1.3828 79.2305 0.000991 
P 0.3150 0.6103 34.9664 1.000000 
S 1.3948 1.8975 108.7215 0.000991 
S0.4254E 38 3.1416 180.0000 0.000991 
MINIMA IN THE PASSBAND (P) 
MINIMA IN THE STOPBAND (S) 
BAND S-DOMAIN Z-DOMAIN MAGNITUDE 
IN RAD IN DEGREE 
P 0. 0. Us 0.999940 
S 0.7495 1.2863 73.7015 we: 
P 0.1693 0.3353 19.2128 0.999940 
S 1.0000 1.5708 90.0000 0.000000 
P 0.2852 0.5557 31.8366 0.999940 
S 2.6670 2.4241 138.8922 0. 
) 0.3249 0.6283 36.0000 0.999940 
LAYOUT OF THE ROUNDED COEFFICIENTS 
WORDLENGTH 8 
COEF COEFS IR IE ID OCTAL 
B2( 1) 1.00000000= 14,0000000042+*( @ = 0} 2**-100 000000000000 
B2( 2) 1.00000000= 1.00000000*2**( O- OQ) 2**-100 000000000000 
B2( 3) 1.00000000= 1.00000000%24e( 0 =- 0) 2**-100 000000000000 
B1( 1) -~-0.56250000= -0.28125000*2**( 1 - OQ) -2**-100 156000000000 
Btt 2) 0. = 0. *2% x ( 1 - 0) 2**-100 100000000000 
Bi --3) 1.50000000= 0.75000000*2**( 1 —- OQ) 2**-100 060000000000 
BO( 1) 1.00000000= 1.00000000*2**( O- 0) 2**-100 000000000000 
BO( 2) 1.00000000= 1.00000000«%2**( O- 0) 2**-100 000000000000 
BO( 3) 1.00000000= 1.00000000*2%**( O- OQ) 2**-100 000000000000 
"Say & oe -1.25000000= -0.62500000*2**( 1 - OQ) -2**-100 130000000000 
ett 33 -1.06250000= -0.53125000*2**( 1- OQ) -2**-100 136000000000 
ok Ea -~0.96875000= -0.48437500*2**( 1- 0) -2**-100 141000000000 
COt-45 0.80468750= 0.80468750*2**( O- 0) 2**-100 063400000000 
ay i eee 0.47656250= 0.47656250*2**( O- 0) 2**-100 036400000000 
ent 3) 0.25781250= 0525781250«2a8¢ 2:0 = 0) 2**-100 020400000000 
SEARCH OF MINIMUM WORDLENGTH 
IWL EPS-P EPS-S PMAX 
8 0.289635 0.979014 1.023068 
BLOCKS OF SECOND ORDER 
CONSTANT GAIN FACTOR = 0.6750014E-02 
a Both) B1(L) BO(L) Chis) CO(L) 
1 1.00000000 -0.56250000 1.00000000 -1.25000000 0.80468750 
2 1.00000000 0. 1.00000000 -1.06250000 0.47656250 
3 1.00000000 1.50000000 1.00000000 -0.96875000 0.25781250 


OPTIMIZATION OF THE PAIRING AND ORDERING 
STORAGE FOR THE INTERMEDIATE RESULTS ISTOR = 100 


FIXED PAIRING YES 
6.1-28 


Section 6.1 


REALIZED STRUCTURE 


SCALING OPTION 
BY A FACTOR OF POWER TWO 
CHOSEN MAXIMUM OF THE 
OVERFLOW POINTS 


REALIZATION OF THE COEFFICIENTS 


CONSIDERATION OF THE QUANTIZATION 


WORDLENGTH 


(FOR IWLR=100 NO ROUNDING) 


NUM DEN SCALING UNCOR. NOISE 


2 2 0.125000 0.28031E 02 0. 


a 3 0.125000 0.65466E 02 OQ. 


1 1 0.250000 0.64145E 02 O. 
TOTAL NOISE 6. 457645 03. 7. 
ABSOLUTE OUTPUT NOISE ANP = 
SCALING AT THE OUTPUT SCA = 
RELATIVE OUTPUT NOISE RNP = 


i 
Le 


RELATED TO MAX /H/ 


INNER WNOISE FIGURE RIN = 


i 
an: 


RELATED TO MAX /H/ 
ENTRANCE NOISE FIGURE REN = 


RELATED TO MAX /H/ = 1 


ADDITIONAL WORDLENGTH DELTA W = 


STRUCTURE WITH SEPARATE SCALING FACTORS B2(L) 


FOR STRUCTURES 21 TO 28 B2(0) = GAIN FACTOR 


BLOCKS OF SECOND ORDER 


CONSTANT GAIN FACTOR = 0.1000000E 


ISTRU = 16 
ISCAL = 2 
YES 
SCALM = 1.000 
YES 
IWLR = 8 
i ie Fs 
COR. NOISE DC-OFFSET CHA CO-QUAN 
0. | 3: wit 
0 0. BIT 
) 0 BIT 
3 4 BIT 
3 +BiF 
0. 3 2: Bit 
3 + Bit 
0 O-BaAt 
2 So -Bit 
2 ie 8 ey 
0% a 2 BIT 
3 4 BIT 
0 : BIT 
3 4;,BiT 
3 7 Bit 
0. 
ix = 0.1576E 03 
0.46048E 01 (Q*Q) = 6.63.08 
0.59205E 00 
0.13137E 02 (Q*Q) ¢ 436 382DB 
0.15764E 03 (Q*Q)/12 = 21.98 DB 
0.25892E 00 
/ 3.484/ = x 
01 
C1(L) CO(L) 


B2(L) B1(L) BO (L) 


0.12500000 0 


1.00000000 


L 

1 ‘ 

2 0.12500000 1.50000000 1.00000000 
3 0.25000000 -0.56250000 1.00000000 


[1] CHA chosen characteristic for the nonlinearities N1...N5. 
CO-QUAN actual wordlength reduction in the nonlinearities N1...N5; this number of bits to be cut off is determined by the 


minimum actual coefficient quantization step size. 


-1.06250000 0.47656250 
-0.96875000 0.25781250 
-1.25000000 0.80468750 
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3.3 Optimized design of a bandpass filter 


The following examples illustrate some further possibilities of DOREDI. First, the input for the optim- 
ization of a bandpass filter is given. 


3.3.1 Specification of the Tolerance Scheme 


*0310 Bandpass 

*0330 Order n.== 10 

*0343 Normalized passband cutoff frequencies; aes 7/10; 0 ies 27/10 
*0344 The normalized stopband cutoff 


frequencies are chosen automatically, 
taking into account the default 
degree extension of 0.2; 


*0352 Tolerated passband attenuation ap = 0.2 dB 
Minimum stop band attenuation as = 40 dB 
3.3.2 Chosen Realization 
*0420 For all the coefficients one common range for the fixed point representation is 


chosen, according to the greatest magnitude of the actual coefficients. 


*0240 Pairing and ordering of the numerators and denominators of the second-order 
blocks will be done by dynamic programming. To avoid small overflows, 
evoked by the rounding error of the state variables, the chosen maximum of 
the scaling is set to SCALM=0.99. 


*0510 The output noise will be optimized for the second canonic form with only one 
nonlinearity after the summation (structure 23). 


*0520 To avoid limit cycles, random rounding is used within the recursive loop 
(JJICNO=7). Only at the end of the cascade two’s complement rounding is 
assigned (JJICNO=1) 


3.3.3 Analysis of the Designed Filter 


*0630 After the optimization of the described filter a printer plot of the magnitude 
*0640 and the attenuation will show the behavior of the frequency response for the 
coefficient set with final wordlength. 


3.3.4 Necessary Input Data Cards 


//88 

*0240 0.99 

*0310 3 

*0330 10 

*0343 0.31416 
*0344 0.62832 

*0352 0.2 40. 
*0420 3 

*0510 23 

*0520 107 

//AN 

*0630 T 

*0640 T 

LF ET 


3.3.5 Line Printer Output of the Designed Filter 
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VERSION v005 === 


DATA INPUT 


//88 
* 2 40 0 0.990000E 00 QO. on 
. i ees 7 * 2 0 es ox 
* c 10 0 Oo: 0: 
= 3°. 43 0 0. 0.314160E 00 QO. 
* 3 44 0 0.628320E 00 0. ae 
* 3-32 0 0.200000E 00 0.400000E 02 Oi 
- 4 20 5 0 Oe us 
* a 49 23 0 G; 0. 
* > 28 107 0 0. 0. 
// AN 
TOLERANCE SCHEME 
FILTER-TYPE 
BANDPASS 
’ 
APPROXIMATION 
ELLIPTIC 
NORM. CUTOFF FREQ. 0. 0.314160 0: -628320 0. 
CUTOFF FREQ. S-DOM. QO. G. 158385 0.324921 o’. 
PASSBAND RIPPLE(S) 0.022763 0.2000 DB P. = Eoated 
STOPBAND RIPPLE(S) 0.010000 40.0000 DB 
FILTER DBGREE ASSIGNED TO ORDER 10 
MIN. FILTER DEGREE 4.1667 
DEGREE EXTENSION 0.8333 
CHOSEN FILTER DEG. 5 
FOR THE REFERENCE LOWPASS 
ACTUAL FILTER DEGREE 10 
BOUND PAIR OF THE DESIGN PARAMETER C 2<196)° 2h Bs 9.6821 
REALIZED 
NORM. CUTOFF FREQ. 0.251463 0.314160 0.628320 O57 13302 
OPTIMIZATION IS TERMINATED AFTER THE 4. STEP 
MAXIMUM NUMBER OF STEPS 
BLOCKS OF SECOND ORDER 
CONSTANT GAIN FACTOR = 0.8861927E-02 
L B2(L) B1 (L) BO(L) oH 2S Me CO(L) 
1 1.00000000: ~1°°410135625 1.00000000 -1.52148437 G2 333599375 
2 1.00000000 -1.09375000 1.00000000 -1.46679687 0. FEST 5625 
3 1.00000000 is -1.00000000 <1, 5) 220003 0.74218750 
Sy 1.00000000 -1.96484375 1 .00000000 =1 JS 939063 0.87500000 
S +. 00000000 = =1.93945313 1.00000000 =T. 86085937 0.96679687 


OPTIMIZATION OF THE PAIRING AND ORDERING 


STORAGE FOR THE INTERMEDIATE RESULTS 


ISTOR = 
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FIXED PAIRING NO 
REALIZED STRUCTURE iSTRU = 23 
SCALING OPTION ISCAL = ie 
BY A FACTOR OF POWER TWO NO 
CHOSEN MAXIMUM OF THE 
OVERFLOW POINTS SCALM = 0.990 
REALIZATION OF THE COEFFICIENTS 
CONSIDERATION OF THE QUANTIZATION YES 
WORDLENGTH IWLR = 11 


(FOR IWLR=100 NO ROUNDING) 


NUM DEN SCALING UNCOR. NOISE COR. NOISE 


DC-OFFSET CHA CO-QUAN 


-49324E-03 


-49324E-03 
.24329E-06= 


(Q*Q) 


1 3 0.103516 0.43090E 02 OO. 0. 

é 1 0.558594 0.60455E 02 OO. QO. 

2 2 0.250000 0. 327078 Aa ~ 0. 0. 

“ 5 0.550781 GO, 517108 03 0. 0. 

3 Se 0.679688 0.16365E 03 0. 0. 

0 0 1.583008 0.10204E 01 O. 0 

0 

TOTAL NOISE 0.81743E 03 0. 0 
ABSOLUTE OUTPUT NOISE ANP = 0.66756E 02 
SCALING AT THE OUTPUT SCA = 0.98994E 00 
RELATIVE OUTPUT NOISE RNP = 0.68119E 02 


RELATED TO MAX /H/ = 1 


INNER NOISE FIGURE RIN = 0.81743E 03 


il 
owe 


RELATED TO MAX /H/ 
ENTRANCE NOISE FIGURE REN = 0.11543E 00 
RELATED TO MAX /H/ = 1 


ADDITIONAL WORDLENGTH DELTA W = / 4.7597 = 
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(Q*Q) * 


(Q*Q)/12 = 


5 


bil 
BIT 
BIT 
BIT 
BIT 


oO O'O' oS 
ow 2: oO © 


BIT 
BIT 
BIT 
BIT 
Biv 


oN OO O 
Oo oo ©'o 


BIT 
BIT 
BIT 
Bie 
Bar 


oS ac 
oMOO0O Oo 


BIT 
BLT 
BIT 
BIT 
BIT 


Ss Oxo 
oak 
oO 2 ao © 


BIT 
BIT 
BIT 
BIT 
BiT 


or © a © 
ean 
Cte co oO © 


Bir 
BIT 
BIT 
BIT 
BIT 


ooo°o=— 
OO Core). 6S 


0.8174E 03 


18.24 DB 


19.33 DB 


29.12. DB 


Chapter 6 


Section 6.1 


ANnNuVHNUWVHnNHUVWNUVMN'D 


BLOCKS OF SECOND ORDER 


CONSTANT GAIN FACTOR 


OP WH 


OPTIMIZATION IS TERMINATED AFTER 


- O00 OO 


B2(L) 


» 92809375 
-25000000 
55078125 
-67968750 
. 58300781 


= 0.1035156E 00 


B1(L) 


. 78770447 
~ 27343750 
-06821442 
. 33547974 


THREE UNSUCCESSFUL STEPS 


CHOSEN DESIGN PAR. 


UTILIZATION OF THE PASSBAND DELTA P 
STOPBAND DELTA S 


CX = 


— © © OO © 


0.0617819 


POLES AND ZEROS IN THE Z-DOMAIN 


CONSTANT GAIN FACTOR = 


NUM. 


NONMNNN ND 


 CeCooc°o 


. 760564 
. 731746 
- 784508 
.887864 
-940917 


POLES 
+-J* 
+-J* 
+-J* 
+-J* 
+-J* 


BLOCKS OF SECOND ORDER 


CONSTANT GAIN FACTOR = 


OPWNH 


EXTREMES OF THE MAGNITUDE OF THE 


— —_— 2 — 


B2(L) 


B1(L) 


0.910673 


- 594928 
.498722 
.351874 
- 292069 
- 286555 


OO © O.O2 


0.910673 


.00000000 
.00000000 
-00000000 
.00000000 
.00000000 


Ts 
wee 

0. 
his 
=~t.. 


40973199 
09325187 


96418508 
93972760 


BO(L) 


- 929859375 
-25000000 
~J00 78125 
-67968750 
- 58300781 


THE 6. 


(=) 


ee 
® 


7E-02 


NUM. 


NN? ANN 


7E-02 


BO(L) 


.00000000 
-00000000 
.00000000 
-00000000 
.00000000 


Cs 


C1(L) 


57246563 
-52148437 
-46679687 
.88085937 
- 77539063 


i) 


«0013992 
.0091129 


| 
oo -0 0 


. 704866 
- 546626 
.000000 
.000000 
- 962093 
.969864 


6.14692 PERCENT 
91.12871 PERCENT 


ZEROS 


+-J* 
+-J* 
+-J* 
+-J* 
+-J* 
+-J* 


C1 (L) 


MAXIMA IN THE PASSBAND (P) 
MAXIMA IN THE STOPBAND (S) 
BAND S-DOMAIN 


SoDooocooc oo Cc oOo 


- 1606 
-4071 
-1804 
~4495 
2269 
-8845 
2852 
.0582 
2205 
-1145 
-1264 


Z-DOMAIN MAGNITUDE 
IN RAD IN DEGREE 
0.3184 18.2449 1.000001 
ee ee E 44.3070 6.009773 
0.307% 20.4577 1.000000 
0.8449 48.4102 0.009113 
0.4462 25.5626 1.000000 
1.4483 82.9829 0.009113 
5556 31.8355 1.000000 
Oo. 1162 6.6600 0.009113 
0.6203 39.9398 1.000000 
0.2280 13.0618 0.009113 
0.2515 14.4078 0.009713 
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-52112896 
-46349257 
- 56901932 
«Liste FOO 
.88183418 


oo oo © 


-4105675 


©. 2:'0o © © 


CO(L) 


- 74218750 
©93359375 
» 78515625 
-96679687 
.87500000 


0.709340 
0.837377 
0. 

0. 
0.188399 
0.243648 


CO(L) 


~- 93239814 
. 78417628 
- 13926776 
.87360604 
. 96743870 


TRANSFER FUNCTION (COEFS NOT ROUNDED) 
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MINIMA IN THE PASSBAND (P) 
MINIMA IN THE STOPBAND (S) 


BAND S-DOMAIN Z-DOMAIN MAGNITUDE 
IN RAD IN DEGREE 

P 0.1584 9 es tah 18.0000 0.998602 
S 0.4161 0.7886 45.1813 a; 

P 0.1676 OC. 3320 19-0257 0.998601 
S 0.5414 0.9925 56.8642 i. 
P 0.2003 0.3954 22.0550 0.998601 
SO0.8507E 38 3.1410 180.0000 0.000000 
P 0.2569 0.5029 28.8158 0.998601 
S G. 0. 0. 0. 
P 0.3071 0.5959 34.1450 0.998601 
S 0.0951 GC. t695 10.8594 0. 
P 0.3249 0.6283 36.0001 0.998601 
S a 3237 0.2461 14.1019 0.000000 
LAYOUT OF THE ROUNDED COEFFICIENTS 

WORDLENGTH 11 

COEF COEFS IR IE 

B27) 0.54687500= 0.27343750*2**( 1 - 0) 
B2t 2) 0.25000000= 0.12500000+24%( 1 = -90) 
Bat 3) 0.55468750= 0.277345 75*2e"( 4 = 0) 
Bat 4) 0.69140625= 0.34570312*2**( 1- QO) 
B2¢ -S) 1.60546875= 0.80273438*2**( 1 - 0) 
ett 43 -0.77148437= -0.38574219*2**( 1 - QO) 
Btt-2) =0.27343/50=. =0.136/T8 754248 “1 = 0) 
BT 3) -1.07617188= -0.53808594x*2**( 1-. QO) 
B1( 4) -1.35742187= -0.67871094*2**( 1- 0) 
Bt -5) 0. = OG. x2ee{--7- = - 2) 
Be 4) 0.54687500= 0.273437504#2"*( - +. -—- 0) 
BO 2) 0.25000000= 0.12500000*2**( 1 - QO) 
BO{ 3) 0.55468750= 0.27734375*2**( 1- QO) 
BO( 4) 0.69140625= 0.34570312*2**( 1- 0) 
BO( 5) -1.60546875= -0.80273438*2**( 1 - 0) 
oe a -1.56835937= -0.78417969x*2**( 1- QO) 
ett 23 -1.52148437= -0.76074219*2**( 1- 0) 
as ee D -1.46289063= -0.73144531*2**( 1 - 0) 
Crt. @) -1.88085937= -0.94042969x2**«( 1- QO) 
Cit 3) -1.77539063= -0.88769532*«2**( 1 - 0) 
fot 4) 0.74023438= 0.37011719*%2**( 1 —- 0) 
Cot 2) 0.93164062= 0.46582032*2**( 1 - Q) 
Cot 3) 0.78320312= 0.39160156*2**( 1 - 0) 
CO( 4) 0.96679687= 0.48339844*2**( 1 - 0) 
Cu) 0.87304687= 0.43652344*2**( 1 - 0) 
SEARCH OF MINIMUM WORDLENGTH 

IWL EPS-P EPS-S PMAX 

Ea i 0.557565 0.991198 0.984173 


BLOCKS OF SECOND ORDER 


CONSTANT GAIN FACTOR = 


0.1058192E 00 


OPWNH A 


-oO0 00 


B2(L) 


- 54687500 
- 25000000 
- 55468750 
-69140625 
-60546875 


B1 (L) 


. 77148437 
- 27343750 
.07617188 
~ 35742187 


BO(L) 
.54687500 
.25000000 
-55468750 
-69140625 
.60546875 


= COE Ce 


6. 


1-34 


2**-100 
2**-100 
2**-100 
2**-100 
2**-100 


~28*=100 


-2**-100 
-2**-100 
-2**-100 

2**-100 


2**-100 
2**-100 
2**-100 
2**-100 
-2**-100 


-2**-100 
-2**-100 
-2**-100 
-2**-100 
-2**-100 


2**-100 
2**-100 
2**-100 
2**-100 
2**-100 


CTia 


«96835937 
-52148437 
- 46289063 
- 88085937 
- 7 7539063 


OCTAL 


021400000000 
010000000000 
021600000000 
026100000000 
063300000000 


147240000000 
167200000000 
135440000000 
124440000000 
100000000000 


021400000000 
010000000000 
021600000000 
026100000000 
114500000000 


115640000000 
117240000000 
121140000000 
103640000000 
107140000000 


027540000000 
035640000000 
031040000000 
036740000000 
033740000000 


oo © & © 


CO(L) 


- 74023438 
-93164062 
« 78320312 
- 9667/9687 
- 87304687 
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REALIZED STRUCTURE 


SCALING 


OPTION 
BY A FACTOR OF POWER TWO 


CHOSEN MAXIMUM OF THE 
OVERFLOW POINTS 


REALIZATION OF THE COEFFICIENTS 


CONSIDERATION OF THE QUANTIZATION 


WORDLENGTH 
NUM DEN SCALING 
1 3 1.000000 
2 1 -000000 
> 2 -000000 
“ 5 -000000 
3 4 -000000 
0 0 -000000 


TOTAL NOISE 


ABSOLUTE OUTPUT NOISE 
SCALING AT THE OUTPUT 


RELATIVE OUTPUT NOISE 


RELATED TO MAX /H/ 


INNER NOISE FIGURE 


RELATED TO MAX /H/ 


ENTRANCE NOISE FIGURE 


RELATED TO MAX /H/ 


ADDITIONAL WORDLENGTH 


| 


ISCAL 


SCALM 


ISTRU = 


IWLR = 


YES 


1 


1 


(FOR IWLR=100 NO ROUNDING) 


0.42382E 


0.63044E 


0.336792 


0.52348E 


0.16769E 


0.10324E 


0.83131E 


ANP 


SCA 


RNP 


RIN 


REN 


DELTA W 


UNCOR. NOISE 


02 


02 


02 


03 


01 


03 


-67100E 


.98417E 


-69276E 


-83131E 


-11864E 


COR. NOISE 


02 


00 


02 


03 


00 


/--. 4, 159/-= 
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DC-OFFSET CHA CO-QUAN 


0.19845E-02 


0.19845E-02 


BIT 
BIT 
BIT 
BIT 
BIT 


Oo 1) 0 :o © 
OOo: Oo'O 


BIT 
Sit 
Bit 
Bie 
BIT 


o~]O © © 
oMOOOO 


BIT 
(8B 
BIT 
BIT 
BIT 


Os] O'S 
oMOO0O Oo 


BIT 
= It 
BIT 
BLT 
BIT 


Orns CO Ore 
©o'0 Oo o'@ 


BIiTt 
BIT 
Bit 
BLT 
BIT 


on OOO 
ars i © | ae J xe J a 


BIT 
BIT 
BIT 
BIT 
BIT 


ooaoo0o = 
© 'O') & '©''00 


0.39384E-05= 0.8313E 03 


(Q*Q) 


(Q*Q) 


(Q*Q)/12 = 


5 


9 


18.27 D8 


18.41 DB 


29.20 DB 
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BLOCKS OF SECOND ORDER 


CONSTANT GAIN FACTOR = 0.1058192E 00 

L B2(L) B1(L) BO (L) C1 (L) CO(L) 

1 0.54687500 -0.77148437 0.54687500 -1.56835937 0.74023438 
2 0.25000000 -0.27343750 0.25000000 -1.52148437 0.93164062 
2 0.55468750 -1.07617188 0.55468750 -1.46289063 0.78320312 
4 0.69140625 -1.35742187 0.69140625 -1.88085937 0.96679687 
5 1.60546875 0. -1.60546875 -1.77539063 0.87304687 
DATA INPUT 

* eee | See 0 QO. 0. 0. 


MAGNITUDE OF THE FREQUENCY RESPONSE 


MAGNITUDE X Ps 0.400 0.800 
pe wo we wo a a on a a a a a a a a a a rrr 
.279E-02 026 + 
.536E-02 OSt + 
. 749E-02 077 + 
.884E-02 102 + 
.897E-02 128 + 
.729E-02 153 + 
315E-02 179 + 
339E-02 204 + 
779E-02 230 + 
196E-01 255.5% 
273E 00 281 + 
942E 00 306 + 
972E 00 332 + 
973E 00 358 + 
973E 00 383 os 
976E 00 409 +. 
980E 00 434 = 
982E 00 460 +. 
983E 00 485 + 
983E 00 511 +. 
984E 00 536 +. 
+. 

587 +, 
982E 00 613 +. 
965E 00 639 + 
761E 00 664 + 
3712 00 690 + 


oooooococoocoocoooocooeocoecoocoocoecoeoocoocoocoococeococoooeoecoecoeo0o0o0coecoco°ce 
WOO 
oO © 
N 
I 
(st BR seo 
oO oO 
SI srmrwmrosQoaoaoo0od0odod0oaccaccoodcoo0ocd0o0no0cccdcedcd0ccocococceoccoeococoeooeooo0o°o°e°eo°ooe 
84) 
fo) 
NO 


Ve 

Ve 

fo) 
t++eeteeteeeeeeeteetetetete 


~ /32E-02 


.288E-03 3.091 
.144E-03 3.116 
-112E-10 3.142 + 


+ + 


ooo 
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DATA INPUT 
* 6 40 T 0 0. 0. 0. 
ATTENUATION OF THE FREQUENCY RESPONSE 

MAGNITUDE X G. 40.000 80.000 
0 1648. 030. 0 8. SPS See See es ele ou ea ge oe ee aS See FM SS eS Se I 
O0.5T1TE 02 0.026 + 

0.454E 02 0.051 + 

0.425E 02 0.077 + 

0.4118 02 0.102 . + 

0.409E 02 0.128 . + 

0 4278 02 0.153). + 

0.5008 02 0.779 . + 

0.494E 02 0.204 + 

0.422E 02 0.230 > 

0.342E 02 0.255 . + 

O,31398 G2 0.281: + 

0, S16E -00 0.306 + 

0.244E 00 0.332 + 

0.240E 00 0.358 + 

0.236E 00 0.383. + 

0.213E 00 0.409 + 

0.179E 00 0.434 + 

0.158E 00 0.460 + 

0.152E 00 0.485 + 

0.148E 00 0.511 + 

0.140E 00 0.536 + 

0.1416 00 0.562 + 

0.156E 00 0.587 + 

0.158E°00 0.613 + 

0.3108 00 0.639 + 

0.2372 01 0.664 .+ 

0.860E 01 0.690 + 

0.167E 02 0.715 + 

0.255E 02 0.741 * 

0.368E 02 0.766 + 

0.5592 02 0.792 + 

0.424E 02 0.817 * 

0.409E 02 0.843 + 

0.416E 02 0.868 + 

0.434E 02 0.894 + 

0.461E 02 0.919 + 

0.502E 02 0.945 + 

0.574E 02 0.971 + 

0.726E 02 0.996 + 
0.5578 02 1.022 + 

0.508E 02 1.047 + 

0.481E 02 1.073 + 

0.463E 02 1.098 ~ 

0.450E 02 1.124 . + 

0.440E 02 1.149 + 

0.433E 02 1.175 + 

0.427E 02 1.200 .+ 

0.423E 02 1.226 .+ 

0.4196 02 1.252 .+ 

0:416E O02. 1.-277 + 

0.414E 02 1.303 + 

0.413E 02 1.328 + 

0.411E 02 1.354 + 

0.411E 02 1.379 + 

0.410E 02 1.405 * 

0.410E 02 1.430 + 

0.410E 02 1.456 + 
0. 708E 02 3.091... . ‘ : ; ‘ ; * 
0.768E 02 3.116 . : : . ; : P ae. ae ; ‘ 
0.163E 03 3.142 . ° ‘ ‘ . . F : : F ‘ I 
DATA INPUT 

//¥FI 
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3.4 Analysis of a given digital filter 


//AN 

*0110 
*0010 
*0010 
*0010 
*0010 
*0010 
*0010 
*0010 
*0510 
*0520 
*0620 
*0630 
*0650 
*0660 
SIELI 


*0110 


*0620 


*0510 


*0520 
*0630 


The coefficients of the given cascade of second-order blocks are read in directly 
by the data input mode 3 (results of example 3.2). 


For the noise analysis, pairing and ordering will be done in the sequence of the 
coefficient input. For the realization structure 16 (Fig. 9) is assumed again. 


Yet, instead of mathematical rounding, unsymmetrical sign magnitude trunca- 
tion is used for the nonlinearities. 


After the noise analysis a printer plot of the frequency response in the region of 
Q =0.0 to 0.8 is given in the same program section. 


The input data cards for this analysis are given here. 


=). 5625 +0 

0.0 Peo 

oo 10 

~tyao 0.8046875 
~1..0625 0.4765625 
=O). 96075 Ose ro lZ. 
0.8 


3.5 Design and analysis with different parameters 


3.5.1 Filter Design 


*0130 


In the first program activity //SY a lowpass filter will be designed, according to 
the specification in sect. 3.1.1. The results are output on the line printer and 
additionally stored on the disk (channel number KA4) to save the original 
(unrounded) coefficients for further program sections. The input data are 
shown in sect. 3.5.4. 


3.5.2 Noise Analysis of the Designed Filter 


*0530 


In the second program activity //AN the filter parameters and coefficients are 
read in from the disk. Besides the default parameter for the filter realization 
the actual coefficient wordlength is assigned to IWLR=12 bit by the control 
card *0530. (Attention: Don’t mix up the parameter IWLR with IWL for the 
optimization.) 


Scaling is done by the absolute criterion, and the chosen maximum is set to 
SCALM=0.99 (see sect. 3.3.2). For the noise analysis the ordering of the 
second-order blocks will be arranged according to the special input. 


3.5.3 Frequency Response for Rounded Coefficients 


*0530 
*0630 
*0640 


*0110 


The actual coefficient wordlength is defined by the parameter IWLR. To avoid 
a double rounding of the coefficients, the printer plots for frequency responses 
with different coefficient wordlength have to be done in separate program sec- 
tions. 


At the beginning of each activity the original coefficients are read in from the 
disk, initialized by the parameter NIMP=2. 
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*0630 Then, the normalized printer plots (length one page) are output on the line 
*0640 printer for the magnitude and the attenuation of the frequency response. 


3.5.4 Necessary Input Data Cards 


Four separate activities are necessary only in computers where it is illegal to follow a write- disc with a 
read-disc. 


//SY 

eO730 7 

*0310 2G 

*0341 ri 4. 
*0351 0.02 0.001 
//¥FI 

//AN 

*0110 2 
*0530 a 
*0620 T + eS 
*0040 3 1 2 
*0040 3 1 2 
406350 Tf 1 

*0640 T 1 

£7 FI 

//AN 

*0110 2 

*0530 10 
206307 1 

*0640 T 1 

AF ET 
//AN 
*0110 
*0530 
+9630. T 
*0640 T 
S/EL 


4. Considerations for the Installation of the Program on a Special Computer 
4.1 Language 


DORED I is written in FORTRAN and follows ANSI FORTRAN conventions very closely. The param- 
eters and variable names are limited to six characters. Only the DECODE statement, permissible for 
most of the compilers, is used in addition. If DECODE or a corresponding statement is not available 
on the compiler in use, the alternative input format should be chosen for the program section and the 
input data cards (see sect. 2.2). Therefore, the input subroutines INP001, INP002, and INP003 have to 
be changed according to the remarks in the beginning of these subroutines before compilation. Conse- 
quently, for the alternative input format the parameters ACX, EDEG, OMLO and RMIN cannot have 
automatic default values. The defined default values are active as long as the parameter is not explicitly 
redefined by the use of an input card including this parameter; e. eB for the definition of a fixed filter 
degree NDEG=10, with input in the normal format 


*0330 10 
the default degree extension EDEG=0.2 has to be set, for the alternative input format, too. 
*0330 fo <-ee 


4.2 Declarations, depending on the computer used 


All declarations depending on the computer precision and special I/O-handling have to be defined by 
the PORT Mathematical Subroutine Library (see Standards chapter of this book). The functions 
I1MACH, RIMACH, and DIMACH are all required. 


4.2.1 Format for Input of Hollerith Characters 


The format must be modified to accommodate the machine in use. The following example applies only 
to the PDP 11/45. 
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FORMAT statement for the input of 80 
Hollerith characters 


n = number of INTEGER 


(n A m) 


m = number of characters 
encoded by one INTEGER 


n*m = 80 


For the PDP 11/45 


DATA IOFO/2H (4,2H0A,2H2)/,ION/40/ 


4.2.2 Special Parameters for the Design and Optimization 


MAXDEG 
MBL 
IWLMI 
IWLMA 


maximum filter degree 
maximum number of blocks of second order f 

minimum coefficient wordlength for the optimization 
maximum coefficient wordlength for the optimization 
(< wordlength of REAL mantissa) 


4.2.3 Subroutine for SPECIAL OUTPUT 


In order to implement a special output, like a paper tape running on a "hardware filter", the user has to 
replace the dummy subroutine "OUTSPE" by his own. The parameter ISPOUT, see sect. 2.2.1.4, can 
be used to generate several output options. The dummy subroutine is listed as an example below: 


C 
ae sen nn el sgh mcm ci er nn te Se SS ge St Na a a wg es ame 
C SUBROUTINE: OUTSPE 
C DUMMY SUBROUTINE FOR ’SPECIAL OUTPUT’ 
Oni een eee oe ee ap oS we ein ans el ena kien ee + SaaS Serene eke S eee 
C 
SUBROUTINE OUTSPE 
C 
COMMON /CONTR / IPRUN,IPCON,NINP,NOUT,NDOUT,LSPOUT, ISPOUT 
COMMON /CANPAR/ KA1,KA2,KA3,KA4,KA5,LINE 
C 
C DATA TRANSFER BY NAMED COMMON BLOCKS: 
C 
COMMON /FILT / NB,FACT,B2(16),B1(16) ,B0(16) ,C1(16) ,C0(16) 
C 
Cc ISPMAX: MAXIMUM NUMBER OF IMPLEMENTED OPTIONS 
ISPMAX=0 
C 
IF(ISPOUT.LE.0) RETURN 
IF (ISPOUT.GT.ISPMAX) GO TO 900 
C 
C EXAMPLE FOR ISPMAX=3 
C 


+ A change of the parameters MAXDEG and MBL entails a change of all COMMON- and DIMENSION-statements, depending 
on these parameters. 
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SOTTO OG AOA OAaAnNAaAn A 


200 


300 


GO TO (100,200,300) , ISPOUT 
CONTINUE 


OPTION NO. 1 
RETURN | 
CONTINUE 
OPTION NO. 2 
RETURN 
CONTINUE 
OPTION NO. 3 
RETURN 

CALL ERROR (8) 


RETURN 
END 
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A.1 General description of the coefficient realization 


For the realization of coefficients with finite wordlength a general description, including the fixed-point 
and "pseudo"-floating-point realization as well as the realization by differences, is useful. The difference 
realization seems to be appropriate for coefficients in the region of the largest possible numbers, espe- 
cially in connection with the "pseudo"-floating-point implementation [4]. The coefficients of the 
transfer function, 


| bz? + b),z2+ 5b 
H(z) = by [J ————*». (Al) 
hal 2 F C47 F Coy 
are renamed by 
{b9,,51,,Bor»C1»Coa! Mg P= 10)5 . (A2) 
The a,, are divided by Eq. (A3) according to Fig. Al. 
2 va sgnlayy) 
< : + 
Avy 
ir. ~le 
oN 2 Ry vA 
Fig. Al "Pseudo"-floating point and difference realization of coefficients. 
bag 4 i 
fe be ee sen(a',,)2.”. (A3a) 
"pseudo"-floating-point 
i ip =i 
a',, = (a,, — sgn(a,,)2-")2°™ ©» (A3b) 
For the difference realization the exponent ip, is calculated as 
ip, = <log|a,,|> (A4) 
otherwise the exponent is fixed to ip = —100; with the definition 2-!°° = 0. <x> means the smallest 


integer greater than or equal to x. 


Since the realized coefficient a',, has to be in the range —1 < a’,, < 1, ig describes the number range 
for the coefficients with the same index v, 


ig, = <log, max {la,, —2 ” sgn(a,,)|}> (A5) 


The "pseudo"-floating-point realization with no forced normalization is represented by only the positive 
exponent i , 


'p “IR 
—| logs (a:.--senka,)2 )2...2] 
ig, = min (A6) 
.... 
with the maximum exponent ig and the definition [x] meaning the largest integer less than or equal 
to x. In this description with hes —100 and : = 0 the simple fixed-point implementation is 
included. 
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B.1 Characteristics of the nonlinearities 


Both the character and the power of the error at the output of a general transfer system (see Fig. B1) 
depend upon the characteristics of the nonlinearities applied [1], [4]. A nonlinearity may produce 
errors of variance a 2k ) which exhibit no correlation with the signal, DC-offsets of size A,, and errors, 
described by the value e,5), which are correlated with the signal (see Fig. B2). Furthermore, DC- 
offsets and correlated errors are correlated with each other. 


The partial errors can be determined following [4], [7], [8]. The quantization characteristics and the 
error values of the different nonlinearities are given in Table B1. 


ee 


a a \ je 


en enn Ge 
| 
on in ees er ee 
i 
! | 


ie ee 


Fig. B1 General transfer function system. 


correlated with 
the other corresponding sources 


Fig. B2 Model of the nonlinearity. 


B.2 Limitation of the error analysis 


For reasons of computation time, the correlated errors are calculated, here, according to the simpler 
linearized model [8]. Furthermore, during the optimization only error correlations within the second- 
order blocks can be taken into account. In the final analysis, however, all correlations are regarded. 
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Table B1 Characteristics of the nonlinearities. 
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C.1 Fixed pairing 


To reduce the necessary computing time for the optimization of the state variable wordlength, a simple 
heuristic method can be used to get a predetermined combination of poles and zeros [1],[4],[6]. To get 
the fixed pairing, in a first step those poles and zeros are combined which are situated most closely to 
each other. Then, without taking into account poles and zeros regarded before, the following steps 
repeat this procedure. The output of the filter synthesis obeys this rule. 


For the first canonical form (Fig. 2) the paired poles and zeros are ordered directly in the blocks of 
second order (Fig. Cl). By means of a simple consideration this rule can be extended for the second 
canonical structure. The numerator function of one block may be thought of as being combined with 
the denominator function of the following block in an auxiliary block [6]; if the rule, mentioned above, 
is applied to these auxiliary blocks, a predetermined combination for the second canonical form is possi- 
ble, too. The scheme in Fig. Cl explains the different pairings for the two canonical forms. 


j 
. 1 2 3 4 5 1 Numerator 
<= a eine ce + cine pean ee sate ae, < ane, toe eae < 77> 1St canonical form 
a 2 3 4 oF Denominator 
Numerator 


24 canonical form 


Denominator 


N\ auxiliary block > 


Fig. C1 Pairing of poles and zeros. 
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D.1 Parameters for the coefficient optimization 


For a tolerance scheme, given by Np, 05, 5p and 55, a minimum filter degree n,,;, is found by 
Amin ~ f(Qp, Qs, dp, 55) ; 


see [1], [2] and [4]. This n,,;, is not necessarily an integer value. Taking the smallest integer n on 
condition that n 2 n,j, and utilizing the total transition band, an unused margin in the passband or 
stopband tolerances remains. According to the choice of the design parameter C, this margin can be 
partitioned to the passband and to the stopband. For C = C,,,,, the total stopband tolerances are ‘util- 
ized for the approximation, and the passband ripple receives a minimum. On the other hand, for 
C = Cinax the total passband tolerances are utilized [1],[4]. 


Crain = S160 p, Qs, dp, ds, n) 


Ca = f,(Qp, Qs, dp, Ss, n) 


Rounding the coefficients of the transfer function leads to errors not proportional to the passband or 
Stopband ripples, in general. For this reason, the parameter C can be used as an additional variable for 
the design of digital filters with rounded coefficients. 


In DOREDI, the optimization of C (Cin < C < Cmax) is done by a trial and error procedure such 
that the required wordlength of the rounded coefficients is small [3], [4]. 


Finally, the parameter C is normalized to the border values C,,,, and C,,,, in order to get a uniform 
variation range in different filter designs: 

logig C — logig Ci 

i. ee 810 810 “min 


ee eS. 1 


6.1-47 


IIR Programs 


Error code: 


WO onrnnnh wd 


Chapter 6 


Appendix E 


Error diagnostics 
Description 


Unexpected input code 

Input data card out of sequence 
Undefined program section name 
Illegal input data 

Parameter out of defined range 
Sampling frequency not defined 
Sampling frequency multi defined 
Not implemented 

Meaningless input data 

Lower bound greater than upper bound 
Missing //FI card 

No input data available 


Filter degree given by input data too small — filter 
degree chosen automatically or program aborted 
Coefficient wordlength out of range 


Not defined filter type 

Not defined approximation type 

Cutoff frequencies not in ascending order 

Missing or illegal definition of the cutoff frequencies 
Missing definition of the tolerances 

Necessary filter degree too high 

Missing filter coefficients 


System not stable 

No fixed pairing possible 
Zeros not on the unit circle 
More than one pole at z = 0 


Option not implemented — Stop 
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Control data input cards 


CLASS 01 


*0110 bb, NINP 

*0120 bb, NOUT 

*0130 LDOUT 

*0140 LSPOUT,ISPOUT 


CLASS 02 


*0210 LWLF,ITERM,ACX,ACXMI,ACXMA 
*0220 bb, ITERM1 

*0230 LPAIRF,ISTOR 

*0240 LSEQ,ISCAL,SCALM 


CLASS 03 


*0310 bb, ITYP,SF 

*0320 bb, [APRO,ACX 

*0330 LDEGF,NDEG,EDEG 
*0341 bb,bb,FR(1),FR(2),SF 
*0342 bb, bb, FR (3), FR (4) ,SF 
*0343 bb,bb,OM (1),0M (2) 
*0344 bb,bb,OM (3),0M (4) 
*0345 bb, bb,S(1),S(2) 

*0346 bb, bb,S(3),S(4) 

*0347 bb,bb, VSN,VD,A 
*0351 bb,bb, ADELP,ADELS 
*0352 bb,bb, AP,AS 

*0353 bb,bb,P,AS 

*0360 bb,NORMA 

*0370 LSOUT 


CLASS 04 


*0410 LSTAB,IWL 
*0420 RCOINP,JRCO 
*0430 ECOINP,JECO 
*0440 DCOINP,JDCO 
*0450 bb, JMAXV 
*0460 bb, JTRB2 


CLASS 05 
*0510 LPOT2,ISTRU 


*0520 CNOINP,JCNO 
*0530 LNCO,IWLR 


CLASS 06 


*0610 LWL,IWL 


*0620 LSEQ,ISCAL,SCALM 


EQIIR 


*0630 LNOR,IPAG,OMLO,OMUP,RMAX 
*0640 LNOR,IPAG,OMLO,OMUP,RMAX 


*0650 LNOR,IPAG,OMLO,OMUP 
*0660 LNOR,IPAG,RMIN,RMAX 
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Program for Minimum —p Synthesis of 
Recursive Digital Filters 
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Institut de Microtechnique 
Universite de Neuchatel 
Rue de la Maladiére 71 

2000 Neuchatel, Switzerland 


1. Purpose 

This program performs the synthesis of recursive digital filters using a minimum p error criterion 
and the method of Fletcher and Powell [2] for function minimization. 
2. Method 


The method is outlined in detail in [1]. The Fletcher-Powell subroutine is taken from [3], but has 
been modified extensively such that the version included here should be used. 


3. Program Description 


3.1 Usage 


The program is self contained with all the necessary subroutines for input and output. The input 
section is also documented with a short description of each of the input parameters. 


The program expects the primary input parameters on unit IND=I1MACH(1) and outputs the 
results on unit IOUTD=IIMACH(2). The final filter coefficients are also output on unit 
IOUTP=I1MACH(3). This could be used for later input, or for producing punched cards, etc. Finally 
in the case of the group delay equalizer, the coefficients of the filter to be equalized are read in from 
unit INP which is set to IIMACH(1). The format expected here is: 


E14.7,14 = CONST, N 
SE14.7. = A0(Q1), A1(1), A2(1), B1(1), B2(1) 


SE14.7. = AO(N), A1(N), A2(N), B1(N), B2(N) 
This same format is used for outputting the final filter coefficients on unit IOUTP=I1MACH(3). 


The program minimizes one of the following three functions: 


Fl = WH()"{X(N)*H(X,J) — FS(I)}**IP (1) 
j= 
F2 = $ WG(){DG(X,J) — DGS(J) — X(N)}**1Q (2) 


J=1 


F3 = ALFA S$; WH()*(X(N—1)*H(X,J) — FS(J)}**1P 
J=1 


NP 
+ (1 — ALFA) } WG(J)*{DG(X,J) — DGS(J) — X(N)}**IQ (3) 
j=1 
where 
Fl is used for magnitude approximation 


IIR Programs Chapter 6 


F2 is used for group delay approximation or group delay equalization 

F3 is used for combined magnitude and group delay approximation 

XxX is the argument vector and contains the zeros and poles in polar coordinates 
J corresponds to a frequency, as discussed below 

FS is the desired magnitude function at the frequency point corresponding to J 

DGS is the desired group delay function at the frequency point corresponding to J 


ALFA is a weighting coefficient between 0 and 1 for the magnitude in the case of com- 
bined approximation. | 


WH (J) is a weighting function for the magnitude at the frequency point corresponding to 
J 


WG (J) is a weighting function for the group delay at the frequency point corresponding 
to J 


H(X,J) _ is the magnitude of the filter which is a function of the argument vector X and 
the frequency point J 


DG(X,J) is the group delay of the filter which is a function of the argument vector X and 
the frequency point J. 


IP,1Q are indices for the magnitude and group delay error respectively. They should be 
positive and even. 


The above functions are computed on a number NK of intervals. These intervals are specified by 
their end points FIK(I) and FIK(I+1) and their number I, I= 1, NK. The number of points on each 
of these intervals is designated by NPK(I), I= 1, NK. The frequency points J are assigned to each 
interval depending on the parameter WEIGHT (I) as follows: 


WEIGHT(I) J corresponds to the frequency 


1 (FIK (I+1)—FIK (1) )*(J-JO—1)/NPK (D+FIK (D equispaced 

2 (FIK (I+1)—FIK (1) *SIN{(J-JO—1)* (PI/2)/NPK (1) }+FIK (D sine half cycle 

3 (FIK (I)—FIK (1+1))*COS{ (J—JO—1)* (PI/2)/NPK (D} +FIK (+1) cosine half cycle 

4 0.5* (FIK (I)—FIK (1+1))*COS{(J—JO—1)*PI/NPK (I) )}+ cosine full cycle 
0.5* (FIK (D+FIK (I+1)) 


where 


= 
JO = > NPK(K) 
K=1 
If NPK(I) = 0, no frequency points are assigned to that interval. This could be used to designate a 
"don’t care" interval. 


The real weight functions WH(J) and WG(J) are constant over any given interval I and are specified 
by the functions WHK(I) and WGK()), i.e., 


WH(J+JO) = WHK(I) for J = 1,NPK(D 
WG(J+JO) = WGK(I) for J = 1,NPK(@) 


The indices IP, IQ are specified by the input parameters IP(II), IQUI), I] = 1,IPK. They should be 
even. If a large index is to be used a sequence such as 2, 4, 10 is recommended. The largest index 
used should be about 10. 


The argument vector X contains the zeros and poles in polar coordinates. On input it specifies the 
initial position of the zeros and poles, while on output it gives their position at the attained minimum 
of F. The zeros are partitioned into real ones, ones on the unit circle and complex ones, while the 
poles are partitioned into real and complex ones. Real zeros and poles are specified by their radii. 
Complex zeros and poles are specified by their angle expressed as a frequency (i.e., angle in radi- 
ans/(27) x sampling frequency) followed by their radii, in pairs. Finally zeros on the unit circle are 
specified by their angle. The number of real zeros, zeros on the unit circle, complex zeros, real poles 
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and complex poles are specified by NZR, NZU, NZC, NPR, NPC in that order. (Note that only zeros 
(poles) with positive angles are specified, their complex conjugate pairs being understood.) The last 
parameter X(N) is a multiplying constant KO for the magnitude for magnitude approximation and an 
added constant TAUO for the group delay for group delay synthesis or equalization. For the case of 
combined magnitude and group delay equalization X (N—1) is KO and X(N) is TAUO. 


The required magnitude and group delay functions FS and DGS have to be provided by separate 
subroutines. A typical subroutine structure is given with the examples below. In the case of group 
delay equalization, the array DGS(J) is set equal to minus the group delay of the filter to be equalized, 
the latter being calculated on the basis of the coefficients read in on unit INP=I1IMACH(1). 


3.2 Types of Output 


The program essentially computes an argument vector X and a gradient vector G corresponding to 
the minimum of the function Fl, F2, or F3 above. Based on X, the zeros and poles as well as the 
coefficients of the cascade realization of the filter are computed. Optionally the frequency response is 
also given. 


Generally two types of output may be obtained. The first type is printed in 72 column format for 
output on a teletype or printer, and consists of either a short output summarizing the results of the 
minimization (LONG=.FALSE.) or a longer output also giving the progress of the minimization 
(LONG=.TRUE.). Examples of both are given below. 


The second type of output is strictly for debugging purposes, and produces extensive intermediate 
results in 72 column format to monitor the progress of the entire program. This should only be used as 
a debugging aid by persons thoroughly familiar with the program. This output is suppressed by setting 
both DEBFP and DEB to .FALSE. in the input sequence. 


3.3 Description of Parameters 
This is given in the input section of the program and is reproduced here for convenience. 
MODE Type of approximation 


=] magnitude only 

=2 group delay only 

=3 group delay equalization 
=4 magnitude and group delay 


NZR Number of real zeros 

NZU Number of zeros on the unit circle 

NZC Number of complex zeros 

NPR Number of real poles 

NPC Number of complex poles 

XxX Initial parameter vector. Zeros and poles specified in polar coordinates as follows. 


Real zeros first followed by zeros on the unit circle, complex zeros, real poles, and 
finally complex poles. Real zeros (poles) are specified as radiusl,radius2,...; zeros on 
the unit circle are specified as anglel,angle2,...; complex zeros (poles) are specified as 
anglel,radiusl,angle2,radius2,..., where the angles are specificed in Hz as 
phi/(27)*fsample and only zeros (poles) with positive angles are specified, their com- 
plex conjugate pairs being understood. The last parameter(s) is (are): 


KO if mode = 1 
TAUO if mode =2,3 
KO,TAUO if mode =4 


NK Number of intervals 
NPK (1) Number of points on the interval FIK (I)—FIK (I+ 1) 
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WEIGHT(I) Spacing of the points NPK(I) on the I" interval 


1 points equispaced 

2 points spaced on a sine abscissa half cycle 

= 3 points spaced on a cosine abscissa, half cycle 
= 4 points spaced on a cosine abscissa, full cycle 


WFK (1) Relative weighting of the I" interval for the magnitude 
WGK (I) Relative weighting of the I'" interval for the group delay 
FIK (I) Starting point of the I'* approximation interval 
IPK Number of successive indices of approximation 
IP (ID) Successive indices of approximation for the magnitude 
IQ(UD Successive indices of approximation for the group delay 
INV Number of total cycles if no convergence 
Fl Lower frequency limit for the frequency response 
F2 Upper frequency limit for the frequency response 
FSAMPL Sampling frequency 
IZ Number of points for the frequency response (IF IZ < 0 no response is calculated) 
ALFA Relative weighting of the magnitude 

Relative weighting of group delay is 1.—ALFA 

This parameter is used only if MODE=4 
LONG Logical parameter for type of output 

= .TRUE. for LONG output 
= .FALSE. for SHORT output 

LIMIT Number of iterations per cycle before restarting with steepest descent 
EPS Error parameter for FMFP subroutine — typically 10~° 
DEB For debugging printout set = . TRUE. 
DEBFP For debugging printout from FMFP set = . TRUE. 


IER 


This completes the description of the input parameters. 


Used on output to indicate: 


= —]1 repeated failure of iteration 

= 0 convergence 

= | failure of convergence in the specified number of iterations 

= 2 failure of linear search to find a minimum along the direction vector 


If IER = 0 and the results are not satisfactory, the program could be restarted with the final argu- 


ment vector as the initial condition. If INV is greater than 1 when IER = 0, this will happen automati- 
cally. 


3.4 Dimension Restrictions 


The number of parameters in X is restricted to 40. The number of filter sections is restricted to 20. 
The total number of points over which F is computed is restricted to 201. The total number of points 
for the frequency response is also restricted to 201. The number of successive indices IP, IQ is res- 
tricted to 10. The number of intervals FIK is restricted to 10. To change any of these, modify the 
appropriate DIMENSION or COMMON statement(s). 
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4. Examples 


Five examples will be given to illustrate how the program works. For each example we give the 
input sequence, the functions FS and DGS (or the input coefficients of the filter to be equalized in 
example 2), followed by the complete output as specified by the input sequence. 


Example 1 


This example illustrates the synthesis of a recursive low pass filter of order 4. The magnitude of the 
filter is to approximate 1 in the range 0 to 3 and 0 in the range 3.4 to 4. There are thus three intervals, 
namely from 0 to 3, from 3 to 3.4 and from 3.4 to 4, with the second one being a "don’t care" interval 
-- hence NPK(2) = 0. The initial argument vector is given as two zeros on the unit circle specified by 
the parameters 3.6 and 3.8, plus 2 complex poles specified by the parameters 1.6, 0.8 and 2.4, 0.8. 
(Note that only the zeros and poles with positive angles are specified -- the complex conjugate ones are 
understood.) The program output is specified as LONG. The input parameters, the magnitude 
specification function FS and the program output are given in Tables 1, 2 and 3 respectively. 


Example 2 


This example illustrates the group delay equalization of a recursive low pass filter. The coefficients 
of the filter to be equalized are used as input on unit INP=I1MACH(1). They are reproduced in Table 
4, together with the input parameters. LONG output is again specified and is given in Table 5S. 


Example 3 


This example illustrates the approximation of the ideal differentiator FS(FI) = FI on the interval 0 
to 1 (with a sampling frequency of 2). Note that now 3 indices IP(II) are specified in the sequence 2, 
4, 10 and the program should restart a maximum of 4 times if no convergence is attained. The input 
parameters for this example are given in Table 6, the magnitude specification function FS is given in 
Table 7 and the (specified) LONG output is given in Table 8. 


Example 4 


This example illustrates the approximation of a rising group delay function DGS(FI) = 15.0*FI 
using an all pass filter of order 8. The input parameters for this example are given in Table 9, the 
group delay specification function DGS is given in Table 10 and the (LONG) output in Table 11. 


Example 5 


This example illustrates the approximation of a recursive low pass filter with both specified magni- 
tude and a group delay. The magnitude is to be 1 from 0 to 0.3 and 0 from 0.6 to 1.0 while the group 
delay is to be 4 from 0 to 0.5. (The sampling frequency is 2.0.) Thus now there are four intervals; 0- 
0.3, 0.3-0.5, 0.5-0.6 and 0.6-1.0. The second and third intervals are "don’t care" intervals for the mag- 
nitude with WHK(2) = WHK(3) = 0, while the last two are "don’t care" intervals for the group delay 
with WGK(3) = WGK(4) = 0. The input parameters for this example are given in Table 12, the mag- 
nitude and group delay specification functions FS and DGS are given in Table 13, and the (LONG) 
output in Table 14. 


References 


1. A. G. Deczky, "Synthesis of Recursive Digital Filters Using the Minimum-p Error Criterion", JEEE 
Trans. on Audio and Electroacoustics, Vol. AU-20, No. 4, pp. 257-263, Oct. 1972. 


2. R. Fletcher and M. J. D. Powell, "A rapidly convergent descent method for minimization", Comput. 
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3. IBM, "System/360 Scientific Subroutine Package Version III", Program No. 360A-CM-03X. 
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FUNCTION: FS 
FUNCTION FOR SPECIFYING THE REQUIRED MAGNITUDE 


10 


20 


30 


40 


50 
60 


FUNCTION FS(FI) 
COMMON /CM40/ FIK(10), NPK(10), NK 
COMMON /UNIT/ IND, INP, IOUTD, IOUTP, PI, SPM 


DO 60 K=2,NK 
K1 = K - 1 


IF (FI.LT.FIK(K1) .OR. FI.GE.FIK(K)) GO TO 60 


Go TO (10,-.20, 30, 40, 50). x1 


FS = 0.0 
CONTINUE 
RETURN 
END 
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KO = 


INITIAL ARG. 


OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 


0.92072797E-01 


O[PO000 0000000000 000 0000000000 


0. 20112E 01 


F= 0.11195E-02 


NO OF FUNCT 


OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 


oo ooo onro0ocdcdq oc oo 00 6 Oo oO 


IER= 0 


OF FUNCT EVALUATIONS= 


2 


Table 3 


NO OF ITERATIONS= 


VECTOR AND GRADIENT 


NO. 
NO. 
NO. 


0 


-48376090E 
-21596835E 
.14348531E 
-44140776E 
.41605789E 
-15939708E 
.85156908E- 


NO OF ITERATIONS= 28 


X( 1)= 0.90000000E 00 

X( 2)= 0.95000001E 00 

X( 3)= 0.40000000E 00 

X( 4)= 0.80000000E 00 

X( 5)= 0.60000000E 00 

X( 6)= 0.80000000E 00 

X( 7)= 0.92072797E-01 

-2011E 01 ITERATION 
1843E 01 ITERATION 
9440E 00 ITERATION 
9437E 00 ITERATION 
2332E 00 ITERATION 
1010E 00 ITERATION 
7646E-01 ITERATION 
4707E-01 ITERATION 
3230E-01 ITERATION 
2245E-01 ITERATION 
2088E-01 ITERATION 
1844E-01 ITERATION 
1840E-01 ITERATION 
1840E-01 ITERATION 
12995 =01 ITERATION 
1226E-01 ITERATION 
8329E-02 ITERATION 
3661E-02 ITERATION 
3357E-02 ITERATION 
3193E-02 ITERATION 
2968E-02 ITERATION 
2671E-02 ITERATION 
2317E-02 ITERATION 
2179E-02 ITERATION 
1915E-02 ITERATION 
1560E-02 ITERATION 
- 4333802 ITERATION 
.1188E-02 ITERATION 

IER= 1 
EVALUATIONS= 135 

X( 1)= 0.94482371E 00 

X( 2)= 0.97314056E 00 

X( 3)= 0.72813510E 00 

X( 4)= 0.43455667E 00 

X( 5)= 0.76811282E 00 

X( 6)= 0.87526403E 00 

X( 7)= 0.33795486E 00 
.1120E-02 ITERATION 
- 1116E-02 ITERATION 
-1114E-02 ITERATION 
.1040E-02 ITERATION 
- LO23E-02 ITERATION 
. 9362E-03 ITERATION 
.8319E-03 ITERATION 
. 7426E-03 ITERATION 
-6654E-03 ITERATION 
- 6060E-03 ITERATION 
-4856E-03 ITERATION 
. 3/06E-03 ITERATION 
sSTFITE-03 ITERATION 
. 2956E-03 ITERATION 
-2345E-03 ITERATION 
.2041E-03 ITERATION 
-1728E-03 ITERATION 
-1621E-03 ITERATION 
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.86673560E- 
-36474567E- 
-85005024E- 
-71129587E- 
. 5034181 5E- 
-14805895E- 
- 11955116E- 


06 


02 
02 
02 
02 
02 
01 
01 
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OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 


-tooon 0S 
. 1466E-03 
.1423E-03 
oe BE oo 5 es 
1319-03 
. 13698-0353 
s1367E-035 
, taeen-—U3S 
-1366E=03 
. 1366E-03 


ag Oo © 2-0 0'oO © 'O 


F='). 43667503 


NO OF FUNCT 


X( 1)= 0 
xi 232-9 
X( 3)= 0 
X( 4)= 0 
X( 5)= 0 
K( 6)= 0 
X( 7)= 0 


IER= 0 


.85934433E 
.93765757E 
.80925703E 
.54+230380E 
. 78067586E 
.88864766E 
.44070618E 


Table 3 


(Continued) 


ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 


NO. 19 
NOs 20 
NO. 21 
NO. 822 
NO. 23 
NO. 24 
NO. 25 
NO. 26 
NO. 27 
NO’ 28 


NO OF ITERATIONS= 28 


EVALUATIONS= 246 


I 
SO oO ae 


.12731826E-04 
.46204799E-04 
.62992023E-04 
.12864991E-04 
.11647338E-04 
.43512307E-04 
.42195573E-04 


SYNTHESIS OF DIGITAL FILTER WITH SPEC. MAGNITUDE AND/OR GROUP 


THE ERROR CRITERION USED IS MINIMUM 2 FOR THE MAGNITUDE AND 


F= 0.136615 -03 


NO OF FUNCT EVALUATIONS= 


IER= 0 


MINIMUM 0 FOR THE GROUP DELAY 


NO OF ITERATIONS= 56 


246 


COMPUTED ARG. VECTOR AND GRADIENT 


Z PLANE ZEROS 


X( 1)= 0.85934433E 
X( 2)= 0.93 7657375 
X( 3)= 0.80925703E 
X( 4)= 0.51230380E 
X( 5)= 0.78067586E 
X( 6)= 0.88864766E 
X( 7)= 0.44070618E 
ANGLE/2PI*FS 


0.34373773E 01 
0.37506063E 01 


GE 1) 
Gt 2) 
GC 3) 
G( 4) 
G{ 5) 
G( 6) 
Gt 27) 


RADIUS 


0.10000000E 


0.10000000E 


Z PLANE POLES 


ANGLE/2PI+*FS 


0.32370281E 01 
0.31227034E. 04 


CONST= 0.44070618E 00 


AO 


RADIUS 


6. 512303808 
0.88864766E 


NUMERATOR COEFFICIENTS 


0.10000000E 01 
0.10000000E 01 


Al 


0:18078962E 01 
Gi1S617561E 01 


DENOMINATOR COEFFICIENTS 


6.2-8 


I 
5 i <3 J <> TN ce Pag aco Ui ee 


| 
Oo 


01 
01 


00 
00 


.12731826E-04 
.46204799E-04 
.62992023E-04 
.12864991E-04 
.11647338E-04 
.43512307E-04 
.42195573E-04 


A2 


DELAY 


0.10000000E 01 
0.10000000E 01 
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BO 


0.10000000E 01 
0.10000000E 01 


FREQUENCY RESPONSE 


FREQUENCY 


N= 


bath Boeke koko kohooohoo koko hoholoRooloohonoRoNohololoi-i-i-i-n he 


- 1000E 
-2000E 
- 3000E 
-4000E 
. 5000E 
-6000E 
. 7000E 
-8000E 
. 9000E 
-1000E 
-1100E 
-1200E 
.1300E 
-1400E 
.1500E 
-1600E 
.1700E 
-1800E 
. 1900E 
-2000E 
.2100E 
.2200E 
.2300E 
-2400E 
. 2500E 
-2600E 
«2 IDOE 
-2800E 
- 2900E 
-3000E 
. 3100E 
-3200E 
. 3300E 
-3400E 
. 3500E 
-3600E 
. 3700E 
.3800E 
. 3900E 
-4000E 
COEFFICIENTS O 


2 


0.4407062E 00 


0.1000000E 01 O. 
0.1000000E 01 OQ. 


0.99734E 
0.997 36E 
0.99744E 
0.99756E 
0.997735 
0.99795E 
0.99822E 
0.998535 
0.99889E 
0. 999295 
0.99973E 
0.10002E 01 
0.10007E 01 
0.10012E 01 
0.10017E 01 
0.10022E 01 
0  1002Z7E 01 
0.10031E 071 
0.10034E 01 
0. 10035E 01 
0.10034E 01 
OQ. 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
ei 


MAGNITUDE 


10030E 01 


-10022E 01 
- 10009E 01 
-99924E 
SS PR PAD 
-99565E 
995382 
-99836E 
-10048E 01 
-99246E 
. 82696E 
-42377E 
. 13648E 
-19414E-01 
- 16017E-01 
-17455E-01 
- 65969E-02 
-60683E-02 
-15293E-01 
-18610E-01 
DIGITAL FILTE 


2 


Table 3 
(Continued) 


B1 


0.84608650E 00 
0.13718319E 01 


LOSS IN DB 


-23141E-01 
» 22927E-01 
»-22285E-01 
-21217E-01 
-19725E-01 
-17812E-01 
-15486E-01 
- 12756E-01 
-96374E-02 
-61515E-02 
-23305E-02 
«1781-28-02 
-0 .61210E-02 
- 10604E-01 
-0.15111E-01 
-19485E-01 
-0°..23515E=01 
- 26921E-01 
-0.29345E-01 
. 30329E-01 
-0 .29314E-01 
- 25655E-01 
=O. 1869712 -01 
. 79214E-02 
-65813E-02 
.23335E-01 
-37861E-01 
-40248E-01 
-14274E-01 
.41574E-01 
-65750E-01 
-16503E 01 
~-74575E 01 
7 299E 
-34238E 
- 35909E 
-35162E 
.43613E 
-44339E 
. 36310E 
-34605E 
WITH SPEC. 


CI) CED) 5 1S > 


Boo oo oOo 00 0200 oO 0 COO 
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-86908E 
.94610E 
~103713E 01 
—11265E 04 
-12340E 01 
<laodroe 2 
-15004E 01 
.14694E 01 
-12546E 01 
269925 
-53160E 
. 15353E 
-935/9E 
-14902E 01 
-12814E 01 
-10007E 01 
-76557E 
.55644E 
-36320E 
1 I938E 
-45134E-08 
AMPLITUDE AND/OR GROUP DELAY 


0.26245519E 00 
0.78969466E 00 


PHASE 


G. 
-0.32716E-01 
-0.65548E-01 
-0.98614E-01 
-0.13204E 
-0.16594E 
-0.20046E 
=O. eagt oe 
= a27 12 
=), S091SE 
-0.34772E 
-0.38774E 
-0 .42948E 
-0.47320E 
=0.57922E 
-0.56790E 
-0 .61966E 
-0.67499E 
-0.73451E 
=O. 
=o 
=) 
=0 
=i) 
-0 
= 
=i 

0 

0 

0 

0 
~8 
=) 
-0 

0 

0 

0 

0 

0 

0 

0 


79891E 


LPIIR 


GROUP DELAY 


i Boeke kaha kekelelolololooloholoholohoholohonohohonohonofohonoi= 


-41631E 
-41705E 
-41927E 
-42301E 
-42832E 
-43528E 
-44399E 
-45460E 
-46726E 
-48219E 
-49964E 
«519925 
-54342E 
-97060E 
-60204E 
-63843E 
-68065E 
- 12979E 
»7/8722E 
. 85469E 
-93449E 
- 10296E 
-11443E 
-12844E 
-14591E 
- 16834E 
-19846E 
.24188E 
s3TT37E 
-43889E 
-69448E 
.10246E 
-87367E 
. 5961 88E 
-40069E 
~- 32261E 
-28006E 
.25453E 
-23889E 
.23024E 
-22747E 


1807896E 01 0.1000000E 01 0.8460865E 00 0.2624552E 00 
1961756E 01 0.1000000E 01 0.1371832E 01 0.7896947E 00 
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0.4054801E+00 Z 
0.1000000E+01 0.1808052E+01 0.1000000E+01 0.7221175E+00 0.2360188E+00 
0.1000000E+01 0.1963544E+01 0.1000000E+01 0.1344155E+01 0.8002729E+00 


Table 5 
TO = 0.23540250E 02 
F= 0.46328E 01 IER= 0 NO OF ITERATIONS= 0 
NO OF FUNCT EVALUATIONS= 2 


INITIAL ARG. VECTOR AND GRADIENT 


X( 1)= 0.12500000E 00 G( 1)= 0.65278458E 02 
X( 2)= 0.80000000E 00 G( 2)= 0.41243368E 01 
X( 3)= 0.25000000E 00 G( 3)= 0.22710607E 02 
X( 4)= 0.80000000E 00 G( 4)= 0.12409674E 02 
X( 5)= 0.37500000E 00 GC S)= 0. 5576453058° D1 
X( 6)= 0.80000000E 00 G( 6)= 0.13062503E 02 
X( 7)= 0.50000000E 00 G( 7)=-0.14036452E 02 
X( 8)= 0.80000000E 00 G( 8)= 0.15493840E 02 
X( 9)= 0.62500000E 00 G( 9)=-0.68205060E 02 
X(10)= 0.80000000E 00 G(10)= 0.45851613E 01 
X(11)= 0.23540250E 02 G(11)= 0.10770746E-05 

OLDF = 0.4633E 01 ITERATION NO. 1 

OLDF = 0.1453E 01 ITERATION NO. 2 

OLDF = 0.1027E 01 ITERATION NO. 3 

OLDF. = 0.9522E 00 ITERATION NO. 4 

OLDF = 0.9090E 00 ITERATION NO. 5 

OLDF = 0.8530E 00 ITERATION NO. 6 

OLDF = 0.8498E 00 ITERATION NO. 7 

OLDF = 0.8458E 00 ITERATION NO. 8 

OLDF = 0.8437E 00 ITERATION NO. 9 

OLDF = 0.8253E 00 ITERATION NO. 10 

OLDF = 0.7907E 00 ITERATION NO. 11 
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OLDF = 0.7892E 
OLDF = . O.777TE. 
OLDF = 0.7758E 
OLDF = 0.77575 
F= 0.77572E 00 


ee 
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Table 5 
(Continued) 


ITERATION NO. 
ITERATION NO. 
ITERATION NO. 
ITERATION NO. 


IER= 0 


EVALUATIONS= 60 


-69931745E-01 
-74191473E 00 
.21490648E 00 
-74735180E 00 
.39960912E 00 
-75370811E 00 
.50457408E 00 
-76478518E 00 
-65060432E 00 
-78936825E 00 
.23742240E 02 


Q 


>) 


Q 
-OVNO WO WNA UNH WN 


NO OF ITERATIONS= 


1 


Os 


5 


10378439E-02 


.57872886E-03 
- 18699270E-02 
.32778834E-03 
.15411069E-02 
.70288699E-03 
.82738208E-03 
-10225077E-02 
.92509451E-03 
.72606624E-03 
.63875690E-04 


SYNTHESIS OF DIGITAL FILTER WITH SPEC. MAGNITUDE AND/OR GROUP 


THE ERROR CRITERION USED IS MINIMUM 0 FOR THE MAGNITUDE AND 


Fe 0 .77572E 00 


MINIMUM 2 FOR THE GROUP DELAY 


IER= 0 


NO OF FUNCT EVALUATIONS= 60 


COMPUTED ARG. VECTOR AND GRADIENT 


Dd dd Dd Dd OS DS DS DS OS OO 
Seiad. 
—=-OOONDUPWNDN — 


Se oo Oo oo'°Oo © ©&' & 


.69931745E-01 
.74191473E 00 
.21490648E 00 
-74735180E 00 
~2 376091 2E 00 
-75370811E 00 
.50457408E 00 
-76478518E 00 
.65060432E 00 
.-78936825E 00 
.23742240E 02 


Z PLANE 


ANGLE/2PI+*FS 


ZEROS 


LF cas I > TR ce YE > 


-27972698E 00 
.85962593E 00 
.14384365E 01 
.20182963E 01 
.26024173E 01 


ANGLE/2PI+*FS 


O'S Oo 


.2/7972698E 00 
<S59625952 00 
.14384365E 01 
01 Ozs0sm 01 
.26024173E 01 


CONST= 0.25809068E-01 


CO CO, CoG 


PLANE POLES 


CES) CS CS 


QAAAA 


aA 


os. 


Q 
uN ee 
=aAOOON DUP WD 


Q 


RADIUS 


.13478638E 
.~+3380579E 
-13267736E 
~ W30 755675 
-12668359E 


RADIUS 


. 74191473E 
-74735180E 
~FS3TOSTTE 
.76478518E 
. 78936825E 


NUMERATOR COEFFICIENTS 


6.2-11 


| 
oe 2 Co. ©. 2 @ 


NO OF ITERATIONS= 15 


-10378439E-02 
.57872886E-03 
. 18699270E-02 
.32778834E-03 
.15411069E-02 
.70288699E-03 
62 738206E-03 
»t0225077E=02 
.92509451E-03 
.72606624E-03 
.63875690E-04 


DELAY 


NE 


IIR Programs 


Table 5 
(Continued) 

AO Al 
0.10000000E 01 -~0.26309318E 01 
0.10000000E 01 -~0.20890137E 01 
0.10000000E 01 = 117327730E 01 
0.10000000E 01 0.37577558E-01 
0.10000000E 01 0.11545468E 01 
0.10000000E 01 0.18080520E 01 
0.10000000E 01 0.19635440E 01 


0.18167368E 01 
0.17903990E 01 
0.17603281E 01 
0.17097046E 01 
0.16048731E 01 
0.10000000E 01 
0.10000000E 01 


DENOMINATOR COEFFICIENTS 


BO B1 
-10000000E 01 -0.14481634E 01 
.10000000E 071 -0.11667867E 01 
-10000000E 01 -0.64350106E 00 
-10000000E 01 0.27197897 7E-0' 


0.71940067E 00 
0.72211750E 00 
0.13441550E 01 


-10000000E 01 
-10000000E 01 
-10000000E 01 


COS > (> 


FREQUENCY RESPONSE 


FREQUENCY MAGNITUDE LOSS IN DB 
0. 0.99397E 00 0 .52569E-01 
0.1000E 00 0.99402E 00 0. 52095E=01 
0.2000E 00 0.99418E 00 0.50674E-01 
0.3000E 00 0.99445E 00 0.48308E-01 
0.4000E 00 0.99483E 00 0.45000E-01 
0.5000E 00 0.99532E: 00 0.40758E-01 
0.6000E 00 0.99591E 00 0 .355932-91 
0.7000E 00 0.99661E 00 0.29525E-01 
0.8000E 00 0.99740E 00 0 ..22582E-01 
0.9000E 00 0.99830E 00 0.14809E-01 
0.1000EF 01 0.99928E 00 0.62713E-02 
0.1100E 01 0.10003E 01 -0.29378E-02 
0.1200E 01 0.10015E 01 -0.12684E-01 
0.13008 20] 0.10026E 01 -0.22783E-01 
0.1400E 01 0.10038E 01 -0:. 32979E-01 
0.1500E 01 0.10050E 01 -0 .42924E-01 
0.1600E 01 0.10060E 01 -0.52152E-01 
0.1700E 01 0.100695 01 -0 .60041E-01 
0.1800E 01 0.10076E 01 -0.65785E-01 
Q0.1900E 01 0.100798 4 -0 .68356E-01 
0.2000E 01 0.10077E 01 -0.66493E-01 
Q0.2100E 01 0.10068E 01 =0..58735E-01 
0.2200E 01 0.10050E 01 -0.43571E-01 
0.2300E 01 0.100238: 507 -0.19834E-01 
0.2400E 01 0.99857E 00 0.12424E-01 
0.2500E 01 0.99424E 00 0.50145E-01 
0.2600E 01 0.99038E 00 0.83991 98-01 
0.2700E 01 0.98939E 00 0.92664E-01 
0.2800E 01 0.99536E. 00 0.40426E-01 
0.2900E 01 0.10098E 01 -0 .84676E-01 
0.3000E 01 0.98849E 00 0.10056E 00 
0.3100E 01 0.72063E 00 0.28458E 01 
0.3200E 07 0.31031E 00 0.10164E 02 
0.3300E 01 0.95332E-01 0.20415E 02 
0.3400E 01 0. 13550E-01 0.373615 02 
0.3500E 01 0.11 T068=07 Q0.39089E 02 
0.360CE 01 0. 12283E-01 0.38214E 02 
0.3700E 01 0.49781E-02 0.46059E 02 
0.3800E 01 0.35983E-02 0.48878E 02 
0.3900E 01 0.98545E-02 0.40127E 02 
0.4000E 01 0. T27105E=07 0.38341E 02 


6.2-12 


0 
=— 
0 


“Qs 
-0. 
.92243E 00 
.10632E 00 
.11475E 01 
.91600E 
.18501E 00 
12679E 01 
.82968E 
.19654E 
.12446E 01 
.80726E 00 
.29869E 00 
.13748E 01 
.73341E 
.28978E 00 
.13479E-.01 
~68727E 00 
42571E 
.14914E 01 
.63383E 00 
. 383155 
.14591E 01 
.54186E 00 
.58274E 
«'S29en- 04 
.57264E 00 
.44639E 
-15006E 01 
. 451395 
.28369E 00 
.84098E 00 
-13067E 01 
.14220E 01 
.10425E 01 
.68428E 00 
.I3906E 00 
.85404E-08 


0.55043747E 00 


0.55853470E 


0.56807592E 00 
0.58489637E 00 
0.62310224E 00 
0.23601880E 00 
0.80027290E 00 


PHASE 


.10481E 01 
.10184E 01 


79897E-01 
11673E 01 


Chapter 6 


GROUP DELAY 


Sooo o oO ocooCcooOooOo OC Oooo OOO OO OOO OO Kmo OO OC OO COO OO oO OO 2 


sl 327 22 
.13483E 
.13887E 
39S 7E 
.13634E 
-13tSae 
.13098E 
.13474E 
.13948E 
.« LS9S95 
sbaos re 
.13100E 
131268 
« F3612E 
.14078E 
<t39 12s 
.13401E 
129922 
, ts 6on 
-1t38115 
.14265E 
.13945E 
~ 131905 
» 12807E 
<lae275 
.14197E 
.14653E 
<E3:/89E 
.12469E 
.12194E 
.14160E 
s19321E 
<1 tte 
» 196535 
.63978E 
.55386E 
«501:38E 
.46763E 
.44624E 
-43428E 
.43042E 


Section 6.2 


a 
1 


(i> TY Jay aes al aT as Ye a ces: ae a 


gi ae TR ws He | 


COEFFICIENTS OF 


FUNCTION: 


7 


. 2980907E-01 
. 1000000E 
.-1000000E 
. 1000000E 
-1000000E 
. 1000000E 
-1000000E 
. 1000000E 


FS 


01-0. 


DIGITAL FILTER WITH SPEC. AMPLITUDE AND/OR GROUP DELAY 


7 
2630932E 01 


.2089014E 01 


<443e7 1730 04 


,31577565-01 


.1154547E 01 


-1808052E 01 


.1963544E 01 


Table 5 


(Continued) 


. 1816737E 
» 17903995 
. 1760328E 
-1709705E 
. 1604873E 
.1000000E 
. 1000000E 


Oo (Oe OS Oa oS 


Table 6 


ONOGOON-> 


aoc) —s 
OS Ora 


ys 
POWDOOOFRN W 


NO - oO 


ao a © 
eke 


i Mae Mead 


Table 7 


1448163E 01 
1166787E 01 


.6435011E 00 
«21978985 -01 
.7194007E 00 
stan li son OO 
.1344155E 01 


FUNCTION FOR SPECIFYING THE REQUIRED MAGNITUDE 


10 


20 


FUNCTION FS(FIL) 
COMMON /CM40/ FIK(10), 


COMMON /UNIT/ IND, 


INP, 


DO 60 K=2,NK 


K1 = 


Ee 


GO 
FS = 
GO 
FS = 
GO 


K - 


4 


(HE 7 FER (1) 


405. 30s 


40, 


NPK(10), NK 


IOUTD, 


30). K1 


6.2-13 


LOUTP, Pt, 


SPM 


.OR. FI.GE.FIK(K)) GO TO 60: 


oO Oo © © © 


.5504375E 
-5585347E 
. 5680759E 
.5848964E 
.6231022E 
.2360188E 
. 8002729E 


LPHR 


IIR Programs 


30 
40 


50 
60 


CONTINUE 
RETURN 
END 


KO = 


rr 
il 


NO 


INITIAL ARG. 


OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 


QO Oo OC Oo Gee oa oO a © o-O @:o:'a' © 


0.40636079E 00 


0.73584E-03 


F= 0.99983E-04 


NO OF FUNCT 


OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 


OCG) OS OO 1S Ore St ar Gare 


IER= 


OF FUNCT EVALUATIONS= 


0 


Table 7 
(Continued) 


Table 8 


2 


NO OF ITERATIONS= 


VECTOR AND GRADIENT 


G ( 
G ( 
G ( 
G ( 
G ( 


ODMDNIADUPWNH- UOPWD — 


—— er OO 


GS Cy Ce 


0 


.89594565E-08 
.83907518E-02 
.71825904E-02 
.93726298E-02 
.44460990E-07 


NO OF ITERATIONS= 20 


G ( 
G ( 
G ( 
G ( 
G ( 


X( 1)= 0.10000000E 01 

Zt 2)= 0. 

X( 3)= 0.400000G0E 00 

X( 4)=-0.40000000E O00 

X( 5)= 0.40636079E 00 

.7358E-03 ITERATION NO. 
.6962E-03 ITERATION NO. 
.5559E-03 ITERATION NO. 
.3074E-03 ITERATION NO. 
.2654E-03 ITERATION NO 
.2616E-03 ITERATION NO 
.2614E-03 ITERATION NO 
. 2613E-03 ITERATION NO. 
.2612E-03 ITERATION NO. 
.2276E-03 ITERATION NO 
.2251E-03 ITERATION NO. 
.2057E-03 ITERATION NO. 
.2056E-03 ITERATION NO. 
.2029E-03 ITERATION NO 
.1928E-03 ITERATION NO 
.1618E-03 ITERATION NO 
.1610E-03 ITERATION NO. 
.1605E-03 ITERATION NO. 
.1341E-03 ITERATION NO. 
.1180E-03 ITERATION NO. 

IER= 1 
EVALUATIONS=_ 87 

X( 1)= 0.93417000E 00 

X( 2)=-0.37288283E 00 

X( 3)=-0.32908125E-01 

X( 4)=-0.50697450E 00 

X( 5)= 0.38296990E 00 

.9998E-04 ITERATION NO. 
9836E-04 ITERATION NO. 
9728E-04 ITERATION NO. 
7T447E-04 ITERATION NO. 
7266E-04 ITERATION NO. 
7207E-04 ITERATION NO. 
6405E-04 ITERATION NO. 
6369E-04 ITERATION NO 
6308E-04 ITERATION NO. 
5820E-04 ITERATION NO. 
3910E-04 ITERATION NO 
2897E-04 ITERATION NO. 
2036E-04 “ITERATION NO. 
1656E-04 ITERATION NO. 
1597E-04 ITERATION NO. 
.1592E-04 ITERATION NO. 
.1591E-04 ITERATION NO. 


6.2-14 


OAANIANANUNHPWNH-UPWD 


~— "oS 


ic 


ce CO 1c. 


«14521 7105-02 
.22680529E-02 
-16926230E-02 
-32463408E-02 
.35211345E-02 


Chapter 6 


Section 6.2 


Table 8 
(Continued) 
F= 0.15910E-04 IER= 0 NO OF ITERATIONS= 17 
NO OF FUNCT EVALUATIONS= 151 
X( 1)= 0.10000424E 01 G( 1)= 0.30941774E-06 
X( 2) =-0.67870385E 00 G( 2)= 0.67787406E-05 
X( 3)=-0.14765522E 00 G( 3)=-0.22261859E-05 
X( 4) =-0.72136589E 00 G( 4) =-0.75488852E-05 
X( 5)= 0.36585334E 00 G( 5)= 0.96522615E-06 
OLDF = 0.6192E-09 ITERATION NO. 1 
OLDF = 0.6140E-09 ITERATION NO. 2 
OLDF = 0.6140E-09 ITERATION NO. 3 


= 0.61399E-09 IER=-1 


NO OF FUNCT EVALUATIONS= 159 


X( 1)= 0.10000470E 01 G ( 

X( 2)=-0.67867164E 00 G ( 

X( 3)=-0.14766660E 00 G ( 

X( 4)=-0.72140586E 00 G ( 

X( 5)= 0.36587823E 00 G ( 
OLDF = 0.6140E-09 ITERATION NO. 
OLDF = 0.6140E-09 ITERATION NO. 
OLDF = 0.6140E-09 ITERATION NO. 
OLDF = 0.6140E-09 ITERATION NO. 
F= 0.61399E-09 IER=-1 


NO OF FUNCT EVALUATIONS= 169 


G ( 


Qa 


G ( 


X( 1)= 0.10000470E 01 

X( 2)=-0.67867164E 00 

X( 3)=-0.14766659E 00 

X( 4)=-0.72140592E 00 

X(. 5)=90.36587815E 09 
OLDF = 0.6140E-09 ITERATION NO. 
OLDF = 0.6140E-09 ITERATION NO. 
OLDF = 0.6140E-09 ITERATION NO. 
OLDF = 0.6140E-09 ITERATION NO. 
F= 0.61399E-09 IER=-1 


NO OF FUNCT EVALUATIONS= 180 

X( 1)= 0.10000470E 01 
X( 2)=-0.67867164E 00 
X( 3)=-0.14766657E 00 
X( 4)=-0.72140595E 00 
X( 5)= 0.36587810E 00 


G ( 
G ( 
G ( 
G( 
G ( 


NO OF ITERATIONS= 3 


)= 0.39802203E-08 
)=-0.21001670E-08 
)=-0.58512773E-08 
)= 0.14038922E-07 
)= 0.21759798E-07 


1 
2 
3 
4 
5 
1 
2 
3 
os 


NO OF ITERATIONS= 4 


0.39722658E-08 
-0.21268438E-08 
-0.58366352E-08 
0.14067840E-07 
6221716322E-07 


— Se 
AS oe 


PWDHY | OP WD > 


NO OF ITERATIONS= 4 


0.39662328E-08 
-0.21494927E-08 
-0.58250289E-08 
0.14092620E-07 
0.21683346E-07 


OP WN = 
1b a Cae a: Sy 


SYNTHESIS OF DIGITAL FILTER WITH SPEC. MAGNITUDE AND/OR GROUP 


THE ERROR CRITERION USED IS MINIMUM 4 FOR THE MAGNITUDE AND 


MINIMUM 0 FOR THE GROUP DELAY 


F= 0.61399E-09 IER=-1 


NO OF FUNCT EVALUATIONS= 180 


COMPUTED ARG. VECTOR AND GRADIENT 


X( 1)= 0.10000470E 01 
X( 2)=-0.67867164E 00 
X( 3)=-0.14766657E 00 
X( 4)=-0.72140595E 00 
X( 5)= 0.36587810E 00 


Z PLANE ZEROS 


6.2-15 


G ( 
G ( 
G ( 
G ( 
G ( 


NO OF ITERATIONS= 48 


. 39662328E-08 
-21494927E-08 
.58250289E-08 
-14092620E-07 
.21683346E-07 


oot i oil 
i O1'.© <= 


DELAY 


LPUR 


IIR Programs 


Table 8 
(Continued) 


ANGLE/2PI+*FS 


<a Ee Sa 


RADIUS 


0.10000470E 
-0.67867164E 


Z PLANE POLES 


ANGLE/ 2PI*FS 


0. 
iF 


CONST= 0.36587810E 00 


AO 


01 
01 


0.10000000E 
0.10000000E 


RADIUS 


-0.14766657E 
-0.72140595E 


NUMERATOR COEFFICIENTS 


DENOMINATOR COEFFICIENTS 


BO 
0.10000000E 01 
0.10000000E 01 

FREQUENCY RESPONSE 

FREQUENCY MAGNITUDE 
i2 0.14602E-04 
0.2000E-01 0. 19535E-01 
0.4000E-01 0.39078E-@1 
0.6000E-01 0.58635E-01 
0.8000E-01 0.78213E-01 
0.1000E 00 0.97820E-01 
0.1200E 00 0.11746E 00 
0.1400E 00 0.13715E 00 
0.1600E 00 0.15688E 00 
0.1800E 00 0.17666E 00 
0.2000E 00 0.19650E 00 
0.2200E 00 0.21641E 00 
0.2400E 00 0.23638E 00 
0.2600E 00 0.25642E 00 
0.2800E 00 0.27653E 00 
0.3000E 00 0. 29671E- 00 
0.3200E 00 0.3T697E 00 
0.3400E 00 OU. 337295 09 
0.3600E 00 0.35768E 00 
0.3800E 00 0.37814E 00 
0.4000E 00 0.39865E 00 
0.4200E 00 0.41921E 00 
0.4400E 00 0.43982E 00 
0.4600E 00 0.46044E 00 
0.4800E 00 0.48108E 00 
0.5000E 00 0.50172E 00 
0.5200E 00 0.522335. 00 
0.5400E 00 0.54290E 00 
0.5600E 00 0.56340E 00 
0.5800E 00 0.58381E 00 
0.6000E 00 0.60410E 00 
0.6200E 00 0.62424E 00 
0.6400E 00 0.64421E 00 
0.6600E 00 0.66399E 00 
0.6800E 00 0.68355E 00 


Al 


-0.10000470E 01 
0.67867164E 00 


B1 


0.14766657E 00 
0.72140595E 00 


LOSS IN DB 


ooo Oc Cao coo OC COO GO COO CO OOM COCO OOCO OOOO Oe Oo Sa GS 


.96712E 
. 34184E 
-28161E 
. 24637E 
.22134E 
-2019TE 
-18602E 
.17256E 
-16089E 
. 150578 
-14133E 
. 132958 
- b2o28e 
.11821E 
tT T65£ 
~ 185538 
G9TITE 
. 94399E 
-89300E 
. 844695 
-79881E 
» foo Se 
.71346E 
-67365E 
-63556E 
- 59908E 
-56411E 
. 53056E 
-49837E 
.46746E 
-43779E 
.40930E 
.38194E 
. 35568E 
-33046E 


6.2-16 


Oca cgoooecgooe@ooooooanooooocoeocoecooaoooaooOo°o oO 


PHASE 


.15491E 
- TS263E 
-15038E 
.14812E 
.14585E 
.14358E 
.14130E 
.13900E 
. 13669E 
-13437E 
.13202E 
.12965E 
27268 
.12485E 
.12240E 
fi P9935 
.11743E 
.11489E 
vi teen 
109725 
IOFCTE 
.10438E 
.10166E 
-_98887E 
-96071E 
93S209E 
.90301E 
-87345E 
.84340E 
.81288E 
. 78186E 
~/5037E 
.71842E 
-68600E 


01 
00 


00 


00 


A2 


B2 


Chapter 6 


GROUP DELAY 


SCOooooo oO @e2ocoGoO GOO OCOeOoO COO C OOOO CoO moO Soo 6 OO 


-85654E 
. 36860E 
-36013E 
«359232 
1 eIF/ TE 
. 36081E 
- 36233E 
. 36422E 
.36645E 
. 36900E 
-3/7188E 
.37508E 
~3/859E 
. 38242E 
-38657E 
~ 391935 
-39580E 
. 40088E 
-40627E 
-41196E 
-41794E 
~-42420E 
-43073E 
-43751E 
-44452E 
.45174E 
-45914E 
- 46668E 
-47432E 
-48202E 
-48973E 
.49738E 
-50493E 
~ 2 12308 
-51945E 


Section 6.2 


OOO © OOo O'O © 2 @ 2 CoO 


. 7000E 
.7200E 
. 7400E 
.7600E 
. 7800E 
.8000E 
.8200E 
.8400E 
. 8600E 
-8800E 
. 9000E 
.9200E 
- 9400E 
.9600E 
. 9800E 
-1000E 
COEFFICIENTS O 


N= 2 


0.3658781E 00 
0.1000000E 01-0. 
0.1000000E 01 O. 


i) ne) le 


ooOo- oO Oo © OO OO oO © 


NO - oO 
Oooo 


DO PNMOON 


0.70288E 
0.72199E 
0.74089E 
0.75961E 
0.77823E 
0.796875 
0.81570E 
0.83496E 
0. 
0 
0 
0 
) 
0 
0 
0 
F 


85494E 


-87600E 
. 89838E 
«922025 
. 94603E 
-96807E 
. 98422E 
~-99025E 
DIGITAL FILTE 


2 


Table 8 


(Continued) 


1000047E 01 O. 


67867165 00 0. 


0.30623E 
0.28294E 
0.26050E 
0.23882E 
0.21778E 
0.19722E 
0.17694E 
0.15667E 
0. 
0 
0 
0 
0 
0 
0 
0 
R 


13613E 


-11499E 
. 93078E 
~70517E 
.48194E 
-28187E 
«1381 28 
.85114E- 


Table 9 


6.2-17 


01 


OO Oo C1 © OO Oro © O1Ore Oo 


53 158 
-61987E 
. 5861 8E 
292098 
, 947618 
-48268E 
-44723E 
-41107E 
. 37386E 
.33498E 
.29348E 
.24793E 
+ 19653E 
.43773E 
« 74360E- 
-228315- 
WITH SPEC. AMPLITUDE AND/OR GROUP DELAY 


01 
08 


0.1476666E 00 0. 
0.7214059E 00 0. 


Oo oo © ©' O'o O'1O':O'Oo oO oO" 


.52634E 
.53294E 
Pe E> S 
.54563E 
-55220E 
-95967E 
- 9691 9E 
282725 
-60349E 
-63643E 
-68835E 
.76656E 
.87389E 
-99885E 
.11068E 
.11506E 
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Table 10 


Chapter 6 


FUNCTION: 
FUNCTION FOR SPECIFYING THE REQUIRED GROUP DELAY 


DGS 


10 


20 


30 


40 


50 
60 


FUNCTION DGS(FI) 


COMMON /CM40/ FIK(10), 
COMMON /UNIT/ IND, 


DO 60 K=2,NK 


Ki 2k = 


IF (PI.L7. FIK(K1) 


GO TO 
DGS = 
GO TO 
DGS = 
GO TO 
DGS = 
GO TO 
DGS = 
GO TO 
DGS = 


CONTINUE 
RETURN 


1 


M 


END 


TO = 


INITIAL ARG. 


OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 


OF FUNCT 


O'S) CO OG OC: Oi @: 3 OOO: Oo O.0O |S Oo oO: © 


0.50424E 02 


ODN AU PWNH — 
a AL AB ae 
oO Oo 8:'o ao Se 


:320358-01 
. 3260E-01 


INP, 


30:, 40, 


NPK(10), 
IOUTD, 


30); 


NK 


K1 


Table 11 


0.79225496E 01 


IER= 0 


EVALUATIONS= 


. 28000000E 
-83000000E 
-49000000E 
-86000000E 
- 68000000E 
-89000000E 
. 78000000E 
-93000000E 
. 79225496E 


2 


NO OF ITERATIONS= 


VECTOR AND GRADIENT 


ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 


6.2-18 


LOUTP PSs 


SPM 


OANA UNPWNH HH UWMWAUPWDND — 


ee OT lee Oe ie 


OO C'S CS eee 


.OR. FI.GE.FIK(K)) GO TO 60 


0 


. 36053095E 
-39729306E 
.27526696E 
.52982008E 
~87300517E 
s11213860E 
.34945496E 
.54749140E 
. 12665987E- 


06 


Section 6.2 


NO OF ITERATIONS= 36 


QQ 


— QMAAADAAAA 


OCOUOWMDNIAM HPWH HO WWA Uh Wb 


—_ 


SV ee Oe ee 


.30741183E-02 
-12525613E-02 
-49062657E-02 
-27420288E-02 
-21828591E-01 
-47436717E-02 
. 8426871 2E-02 
-63354241E-02 
.17568368E-02 


NO OF ITERATIONS= 10 


Table 11 
(Continued) 

OLDF = 0.3244E-01 ITERATION 
OLDF = 0.3244E-01 ITERATION 
OLDF = 0.3230E-01 ITERATION 
OLDF = 0.3196E-01 ITERATION 
OLDF = 0.3077E-01 ITERATION 
OLDF = 0.3020E-01 ITERATION 
OLDF = 0.2964E-01 ITERATION 
OLDF = 0.2942E-01 ITERATION 
OLDF = 0.2914E-01 ITERATION 
OLDF = 0.2865E-01 ITERATION 
OLDF = 0.2803E-01 ITERATION 
OLDF = 0.2752E-01 ITERATION 
F= 0.27454E-01 IER= 1 
NO OF FUNCT EVALUATIONS= 136 

X( 1)= 0.41772590E 00 

X( 2)= 0.57610147E 00 

X( 3)= 0.62142190F 00 

X( 4)= 0.63898080E 00 

X( 5)= 0.10983226E 01 

X( 6)= 0.75217267E 00 

X( 7)= 0.77566455E 00 

X( 8)= 0.68576224E 00 

X( 9)= 0.86191942E 01 
OLDF = 0.2745E-01 ITERATION 
OLDF = 0.2745E-01 ITERATION 
OLDF = 0.2745E-01 ITERATION 
OLDF = 0.2745E-01 ITERATION 
OLDF = 0.2745E-01 ITERATION 
OLDF = 0.2745E-01 ITERATION 
OLDF = 0.2745E-01 ITERATION 
OLDF = 0.2745E-01 ITERATION 
OLDF = 0.2745E-01 ITERATION 
OLDF = 0.2745E-01 ITERATION 
F= 0.27449E-01 IER= 0 
NO OF FUNCT EVALUATIONS= 176 

X( 1)= 0.41735033E 00 

X( 2)= 0.57608889E 00 

X( 3)= 0.62094770E 00 

X( 4)= 0.63927549E 00 

X( 5)= 0.10985726E 01 

X( 6)= 0.75270749E 00 

X(-7)= 0.77532603E 00 

X( 8)= 0.68637722E 00 

X( 9)= 0.86208143E 01 
OLDF = 0.5880E-02 ITERATION 
OLDF = 0.5097E-02 ITERATION 
OLDF = 0.4842E-02 ITERATION 
OLDF = 0.4310E-02 ITERATION 
6LDF = 0.41248-02 ITERATION 
OLDF = 0.3709E-02 ITERATION 
OLDF = 0.3623E-02 ITERATION 
OLDF = 0.3590E-02 ITERATION 
OLDF = 0.3527E-02 ITERATION 
OLDF = 0.3213E-02 ITERATION 
OLDF = 0.3060E-02 ITERATION 
OLDF:s- 0.2971E-02 ITERATION 
OLDF = 0.2940E-02 ITERATION 
OLDF = 0.2932E-02 ITERATION 
OLDF = 0.2931E-02 ITERATION 
F= 0.29311E-02 IER= 0 


NO OF FUNCT 


X ( 
xf 
X ( 


EVALUATIONS= 254 


OMDNADUPWNHHUOWMNANUPWDND — 


Or Sayre (Com oO > 


» 235099838E-05 
.63836675E-06 
-41995328E-05 
-48311127E-07 
.11442379E-04 
-16899898E-05 
-43452516E-05 
3 18039298 -05 
.64447522E-06 


NO OF ITERATIONS= 15 


1)= 0.40242405E 00 
2)= 0.59617415E 00 
3)= 0.61030855E 00 


6.2-19 


.34703542E-04 
.57860917E-04 
.37734724E-04 
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SYNTHESIS OF DIGITAL FILTER 


THE ERROR CRITERION USED IS 


= 0.29311E-02 


Table 11 


(Continued) 


-66355933E 
«1103 2927E 
«77201411E 
. 76772848E 
-71009994E 
.87403825E 


noeuouw Wow wl 
S''o' oo Gy o' © 


Mr OM OM OM OM 


QAAAAN 


I 
o.oo 2 oc 2 > 


.35411865E-04 
- 78650220E-04 
«1 2829927E-03 
-15171362E-03 
-34973891E-04 
- 19682339E-05 


WITH SPEC. MAGNITUDE AND/OR GROUP 


MINIMUM 0 FOR THE MAGNITUDE AND 


MINIMUM 4 FOR THE GROUP DELAY 


IER= 0 


NO OF FUNCT EVALUATIONS= 254 


COMPUTED 
CONST= 0. 
0 
0 
0 
0 
0 
0 
0 
0 


ARG. VECTOR AND GRADIENT 


X( 1)= 0.40242405E 00 
X( 2)= 0.59617415E 00 
X( 3)= 0.61030855E 00 
X( 4)= 0.66355933E 00 
X{ 5)= 0.11705292728 701 
X( 6)= 0.77201111E 00 
X( 7)= 0.76772848E 00 
X( 8)= 0.71009994E 00 
X( 9)= 0.87403825E 01 
Z PLANE ZEROS 
ANGLE/ 2PI+*FS 
0.40242405E 00 0 
0.61030855E 00 0 
0.89470732E 00 0 
0.76772848E 00 0 


Z PLANE POLES 


ANGLE/ 2PI+*FS 


0.40242405E 00 
0.61030855E 00 
0.89470732E 00 
0.76772848E 00 


47031828E-01 


OQ"; O 


Q 
OMDANANUNLPWDNDH 


RADIUS 


-16773622E 
.15070242E 
«129931818 
-14082525E 


RADIUS 


-599617415E 
- 66355933E 
s7120tt + 1s 
. 71009994E 


NUMERATOR COEFFICIENTS 


6.2-20 


A0 Al 
- 10000000E 01 -0.10123400E 01 
-10000000E 01 0.10237209E 01 
- 10000000E 01 0.24501904E 01 
-10000000E 01 0.20993461E 01 
DENOMINATOR COEFFICIENTS 
BO B1 
- 10000000E 01 -0.35980954E 00 
-10000000E 01 0.45075555E 00 
-10000000E 01 0.14603163E 01 
-10000000E 01 0. 105857632 04 


O'S OO OC Ole S'S 


NO OF ITERATIONS= 61 


.34703542E-04 
-57860917E-04 
.37734724E-04 
-35411865E-04 
- 78650220E-04 
1 26299218 =O 3 
» 1317 13628403 
-34973891E-04 
-19682339E-05 


A2 


.28135440E 
221 Visels 
.16778491E 
13ST 7 sik 


SO Co'cre 


B2 


.35542362E 
-44031099E 
~ 29600115E 
-50424192E 


COS SS 


Chapter 6 


DELAY 


Section 6.2 


FREQUENCY RESPONSE 


FREQUENCY 


. 2000E-01 
-4000E-01 


OS) OO OO) OOOO OOOO) CO S's O'S) |S C'S 
NO 
e)) 
Oo 
Oo 
fe 
a} 
oO 


OEFFICIENTS O 


S 


»4703.183E= 


. 1000000E 
.1000000E 
. 1000000E 
.1000000E 


Soo mo wb ooo 0 00 CO OD Os oO oo Oo 0 Coo 002 0 2 
CO 
Ss 
ro) 
ro) 
ea) 
) 
-) 


MAGNITUDE 


0 
0 
0 
0 
0 
) 
0 
0 
0 
0 
) 
0 
0 
0: 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 


0 
0 
0 
0 
0 
0) 
0 
0 
) 
0 
0 
0 
0 
0 
0 
) 
0 
0 
0 
0 
0 
0 
0 
0 
F 


01 

Ut-0. 
O42). 
O41: 0; 
35 aoe 


-10000E 01 
.10000E 01 
. 1090008 01 
.10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
» 10000E 01 
.10000E 01 
-10000E 01 


10000E 01 


-10000E 01 
.10000E 01 
-10000E 01 
-10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
.10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 
-10000E 01 
.10000E 01 


L 


= 
#0 
=§ 

0 
-0 

0 
-0 


Sco@monD OO COO OOo OC CO Coo eo oO hb oOo © 0 0 O80 0 OC O00 Oo Oo 2a 


ee | 
CO). 


© 5 


DIGITAL FILTER 


4 

1012340E 
1023721E 
2450190E 
2099346E 


iO O'!'O 


.2813544E 
weallt228 
.1677849E 
~ 19831752 


Table 11 


(Continued) 


OS5- 28 DS 


.12943E-06 
.12943E-06 
.12943E-06 
.12943E-06 
.12943E-06 
.12943E-06 
.12943E-06 


.12943E-06 
.12943E-06 


.19414E-06 
.19414E-06 


.19414E-06 
.12943E-06 
.12943E-06 
.19414E-06 
.12943E-06 


.12943E-06 
.12943E-06 
.12943E-06 
.19414E-06 
.19414E-06 
. 38829E-06 
323575 -06 
.19414E-06 
.19414E-06 
~o2397E-06 
.19414E-06 
.12943E-06 
~s230 1B-06 
.19414E-06 
.12943E-06 
.12943E-06 
.12943E-06 
.19414E-06 
.19414E-06 
.12943E-06 
.12943E-06 
.12943E-06 
. 12943E-06 
.12943E-06 
ot ti eeeUus 
.38829E-06 
. 6471 5E-06 
.38829E-06 
51772E-06 


PHASE 


{5963E 00 


.32026E 00 
.48288E 00 
.64857E 00 
.81845E 00 
.99375E 00 
<A TTSEE 07 
-13661E 01 
.15664E 01 
<13631E 01 
-11370E 01 
.89467E 00 
-G35598E 00 
—25178E 00 
.45080E-01 
.28666E 00 
.64506E 00 
-10298E 01 
.14389E 01 
st27268: 05 
.82590E 00 
.36633E 00 
-10375E 00 
.58437E 00 
.10780E 01 
s155252. 04 
- T0T90E 01 
.45890E 00 
.12938E 00 
.74340E 00 
ctarises. U4 
.11174E 01 
-46125E 00 
.20509E 00 
.88765E 00 
.15473E 01 
«611 392-00 
.47784E-01 
739025 00 
-152758 07 
.81401E 00 
.21088E-02 
.84736E 00 
.14170E 01 
—pa7o te 00 
.32665E 00 
2tT1612 04 
—La0ee8 01 
.63709E 00 
.19966E-07 


GROUP DE 


.25380E 
.25459E 
- 226965 
. 26098E 
.26673E 
.27434E 
.28400E 
. 295928 
.31036E 
.32764E 
.34810E 
. 37209E 
<s9995E 
.43182E 
-46772E 
.50716E 
.54908E 
. 99TO6E 
.63246E 
.66885E 
.69887E 
<12S0U VE 
.74012E 
. 75620E 
.77434E 
. 7981 7E 
.82988E 
. 86945E 
-91398E 
.95794E 
.99489E 
- 102092 
.10374E 
-105143E 
.10714E 
.11035E 
.11473E 
.11948E 
.12334E 
- 125555 
.12664E 
128278 
« 1371908 
-13728E 
.14138E 
139978 
131678 
~12012E 
. 109715 
- 102932 
.10063E 


—mofoeleheholohehohohohoonolo foo oRo neko Ronen ek eee eee eee ee eee 


LAY 


02 


WITH SPEC. AMPLITUDE AND/OR GROUP DELAY 


6.2-21 


01-0.3598095E 00 
01. 0..4507555E 00 
01 0.1460316E 01 
01 0.10585785 01 


0.3554236E 
0.4403110E 
0.596001 2E 
0.5042419E 
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i) Me) le 


20.000 0 


OoOp-OOoOCooaoooaoo0aoo0o Oo 


=]-oaoaoqo00coo}}--0ao0> 


NO - oO 


©: dV Viw ooo o O° @ Go oa 


Table 12 


Table 13 


Chapter 6 


FUNCTION: FS 
FUNCTION FOR SPECIFYING THE REQUIRED MAGNITUDE 


10 


FUNCTION FS(FI) 
COMMON /CM40/ FIK(10), NPK(10), NK 
COMMON /UNIT/ IND, INP, IOUTD, IOUTP, PI, SPM 


DO 60 K=2,NK 
K1 = K- 1 


IF (FI.LT.FIK(K1) <OR. PI.GE.FIK(K)) GO TO 60 


gO TO (10, 20, 30, €0,.-S8),- 1 
FS = 1.0 


6.2-22 


Section 6.2 
Table 13 
(Continued) 
GO TO 60 
20 FS = 0.0 
GO TO 60 
30 FS = 0.0 
GO TO 60 
40 FS = 0.0 
GO TO 60 
50 FS = 0.0 
60 CONTINUE 
RETURN 
END 
Cc 
ie SS = SS Se SB ee ww me ew eae ee ee ee eee a eee ee ee ee ee mee ee oe eee we ee ee ee eee ee ee ee ee ee ee 
C FUNCTION: DGS 
C FUNCTION FOR SPECIFYING THE REQUIRED GROUP DELAY 
Cc me a a a ww ww wm @ we wm oe ee we oe wm a a a a eee ee a a a eee ee eee ee 
Cc 
FUNCTION DGS(FI) 
COMMON /CM40/ FIK(10), NPK(10), NK 
COMMON /UNIT/ IND, INP, IOUTD, IOUTP, PI, SPM 
Cc 
DO 60 K=2,NK 
Ki-e & - 4 
C 
IF (FI.LT.FIK(K1) .OR. FI.GE.FIK(K)) GO TO 60 
Cc 
GO TO (10, 20, 20,40; 50),.44 
10 DGS = 4.0 
GO TO 60 
20 DGS = 4.0 
GO TO 60 
30 DGS = 0.0 
GO TO 60 
40 DGS = 0.0 
GO TO 60 
50 DGS = 0.0 
60 CONTINUE 
RETURN 
END 
Table 14 
KO = 0.18284121E-01 TO = 0.35229725E 01 
F= 0.34480E-02 IER= 0 NO OF ITERATIONS= 
NO OF FUNCT EVALUATIONS= 2 
INITIAL ARG. VECTOR AND GRADIENT 
X( 1)= 0.42000000EF 00 G( 1)= 0 
X( 2)= 0.63200000E 00 G{'2)=°6 
X( 3)= 0.85100000E 00 G( 3)= 0 
X( 4)= 0.23800000E 00 G( 4)=-0 
X( 5)= 0.50700000E 00 G( 5)= 0 
X( 6)= 0.10100000E 00 G( 6)= 0 
X( 7)= 0.61900000E 00 G( 7)=-0 
X( 8)= 0.27400000E 00 G( 8)= 0 
X( 9)= 0.62900000E 00 G( 9)=-0 
X(10)= 0.45800000E 00 G(10) =-0 
X(11)= 0.73200000E 00 G(11)= 0 
X(12)= 0.18284121E-01 G(12)= 0 
X(13)= 0.35229725E 01 G(13)= 0 
OLDF = 0.3448E-02 ITERATION NO. 1 
OLDF = 0.2780E-02 ITERATION NO. 2 
OLDF = 0.2675E-02 ITERATION NO. 3 
OLDF = 0.2158E-02 ITERATION NO. 4 
OLDF = 0.1901E-02 ITERATION NO. 5 
OLDF = 0.1798E-02 ITERATION NO. 6 
OLDF = 0.1665E-02 ITERATION NO. 7 
OLDF = 0.1493E-02 ITERATION NO. 8 


6.2-23 


0 


.21118077E-02 
.28182383E~02 
-6365571 5E-03 
 faa/S938E-02 
. 3971151 27E-02 
-20516445E-01 
-42940497E-02 
- 17985253E-01 
-18634771E-01 
-13497053E 00 
.83689370E-01 
-18122631E-06 
.84692147E-08 


LPHUR 
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OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 
OLDF 


| AOA) pe | et | Wart | eat | Ske | hoes Ca Wma | a ee | Dea | ieee | ae ae | Se CE aes | eeu Verma | Ba | teen | Te)? ac | fale | Ge Rat ee | ime | Fat | Waal | UNG! | al | We | | as | ead ei | at | eee | lf 


OOO Cee (1 OS) (O11 Sa SG (Se Ga ies CoS Ce aS OG Ce) Oe O'S O'S (OS 


.1283E-02 
e 1191E-62 
.102Z3E-82 
«0165-03 
.6314E-03 
~91 938-03 
.4512E-03 
.4389E-03 
.4170E-03 
.4020E-03 
.3969E-03 
. 8265-03 
.3754E-03 
sa 1SEO0S 
.3663E-03 
. 3616E-03 
.3543E-03 
. 3468E-03 
-24338-03 
setae o3 
.32 18-05 
.3184E-03 
s3196E8-03 
so1s/n-03 
.3087E-03 
-s0528-03 
<u 19E~0 3 
se790n-03 
2911TE=03 
. 2890E-03 
.2053E-05 
.2834E-03 
26318-0353 
<2623E-05 
52815E-03 
.2199E=03 
-2192E-03 
.2784E-03 
hite-Us 
stl te es 
<2/61E-035 
,2idsE-—US 
-2790E-03 
.2746E-03 


F= 0.27332E-03 


NO OF FUNCT 


X ( 
X ( 
X ( 
X ( 
X ( 
X ( 
zi 
+ $i! 
X ( 


-OVOUWONAUPWD — 


0 
0 
0 
0 
0 
0 
0. 
0) 
0 
0 
0 
0 
0 


Table 14 


(Continued) 


IER= 1 


EVALUATIONS= 191 


.24982625E 
-62820711E 
.84752128E 
-15254241E 
<2962/71715E 
-10178863E 


54562625E 


-31A60252E 
- 56832202E 
-50460494E 
.64320938E 
po807192 16E= 
. 308114125 


ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 
ITERATION 


NO OF ITERATIONS= 52 


02 
01 


G( 1)= 

G( 2)=-0 
G( 3)=-0 
G( 4)=-0 
G( 5)= 

G( 6)=-0 
G( 7)=-0 
G( 8)= 0 
G( 9)=-0 
G(10)= 0 
G(11)=-0 
G(12) =-0 
G(13)= 0 


. 34937362E-02 
-13012964E-02 
~ 3196 76618-0353 
.89132080E-03 
«9420992 35-02 
.81774487E-04 
»49900725E-03 
, $e 35985E-02 
.65854154E-03 
42 1 BOD SIZ6E=02 
~33977294E-03 
.20130444E 00 
.11298486E-03 


Chapter 6 


SYNTHESIS OF DIGITAL FILTER WITH SPEC. MAGNITUDE AND/OR GROUP DELAY 


THE ERROR CRITERION USED IS MINIMUM 2 FOR THE MAGNITUDE AND 


F= 0.27332E-03 


NO OF FUNCT EVALUATIONS= 191 


IER= 1 


MINIMUM 2 FOR THE GROUP DELAY 


NO OF ITERATIONS= 52 


6.2-24 


Section 6.2 


Table 14 
(Continued) 


COMPUTED ARG. VECTOR AND GRADIENT 


. 34937362E-02 
-13012964E-02 
- 31967661E-03 
-89132080E-03 
-97309923E-02 
-81774487E-04 
-49900725E-03 
-13535585E-02 
.65854154E-03 
-21895326E-02 
. 33977294E-03 
-20130444E 00 
.11298486E-03 


X.¢ 1 }= +0. 249826258: .00 ib Ah< 
Ewe 0.62820 7412:-00 Gt 2)s 
X€-3%= 0. 84752 tak 200 G¢23)= 
X (4) = 90:. 15254 249m 100 G(: 4) = 
XC Ses 4). 29623 7E1SE 700 GE;:5)= 
X( 6)= 0.10178863E 00 G( 6)= 
KCoTP Pe 70. 545626258:-00 Gt :7)= 
X( 8)= 0.31160252E 00 Gt @)-= 
x( -Sy= ©. 568322025 00 GE-9o)= 
X(10)= 0.50460494E 00 GA-:0:) = 
X(11)= 0.64320938E 00 G(11)= 
X(12)= 0.29779218E-02 G(12)= 
%C13)= 00. 30811 412E 01 G(13)= 
Z PLANE ZEROS 
ANGLE/ 2PI*FS RADIUS 
0. 0.24982625E 
G. 0.40027819E 
0.62820711E 00 0.10000000E 
0.84752128E 00 0.10000000E 
0.15254241E 00 QO 296277155 
0.:45254241E 200 0.33752181E 
Z PLANE POLES 
ANGLE/ 2PI*FS RADIUS 
0.10178863E 00 0.54562625E 
0.31160252E 00 0.56832202E 
0.50460494E 00 0 .64320938E 


CONST= 


0. 


Z297T19218E-02 


NUMERATOR COEFFICIENTS 


DENOMINATOR COEFFICIENTS 


A0 
0.10000000E 01 =O 
0.10000000E 01 0 
0.10000000E 01 0 
0.10000000E 01 =. 
0.10000000E 01 #0. 
BO 
0.10000000E 01 =O, 
0.10000000E 01 =O; 
0.10000000E 01 O. 
0.10000000E 01 0. 
0.10000000E 01 OG: 
FREQUENCY RESPONSE 
FREQUENCY MAGNITUDE 
Os 0.98200E 00 0 
0.2000E-01 0.98309E 00 0 
0.4000E-01 0.98617E 00 0 
0.6000E-01 0.990715. 00 0. 
0.8000E-01 0.99600E 00 0 
0.1000E 00 0.10013E 01 ~() 
0.1200E 00 0.10061E 01 =( 


-78394464E 


A1 
42526081E 
17748885E 


52580426E 
59900133E 


B1 


10359316E 01 
63414777E 00 
18609760E-01 


LOSS IN DB 


-15780E 00 
.14814E 00 
.12095E 00 


81041E-01 


-34840E-01 
.11414E-01 
.532378-01 
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OS ©) 


0 
-0 
= 
a!) 
0 
-0 
=0 
0 
-0 
0 
=0 
=) 
0 
00 
01 
01 
01 
00 
01 
00 
00 
00 
0 
0 
0 
0 
0 
PHASE 
. 38178E 
-76402E 
.11470E 
-15306E 
122728 
-84362E 


A2 


-10000000E 01 
-10000000E 01 
.10000000E 01 
-87780149E-01 
sA4S92Z097E 02 


B2 


-29770800E 00 
» 32298992E 00 
-41371830E 00 


LPHUR 


GROUP DELAY 


SB ja Ge FAN aD Yin am WY ca i ee BA a) 


-60749E 
.60788E 
-60889E 
.61007E 
-61087E 
-61085E 
-60983E 


IIR Programs Chapter 6 
Table 14 
(Continued) 
0.1400E 00 0.10102E 01 £-0.88524E-01 0.46099E 00 0.60802E 01 
0.1600E 00 0.10136E 01 -0.11694E 00 0.79619E-01 0.60594E 01 
0.1800E 00 0.10160E 01 -0.13830E 00 -0.30054E 00 0.60425E 01 
0.2000E 00 0.10175E 01 -0.15047E 00 -0.67991E 00 0.60351E 01 
0.2200E 00 0.10171E 01 -0.14766E 00 -0.10592E 01 0.60397E 01 
0.2400E 00 0.10139E 01 -0.11948E 00 -0.14391E 01 0.60553E 01 
0.2600E 00 0.10059E 01 -0.51280E-01 0.13214E 01 0.60765E 01 
0.2800E 00 0.99148E 00 0.74290E-01 0.93892E 00 0.60959E 01 
0.3000E 00 0.96885E 00 0.27488E 00 0.55552E 00 0.61062E 01 
0.3200E 00 0.93692E 00 0.56594E 00 0.17188E 00 0.61032E 01 
0.3400E 00 0.89543E 00 0.95935E 00 -0.21119E 00 0.60886E 01 
0.3600E 00 0.84490E 00 0.14639E 01 -0.59313E 00 0.60690E 01 
0.3800E 00 0.78632E 00 0.20880F 01 £-0.97394E 00 0.60544E 01 
0.4000E 00 0.72079E 00 0.28438E 01 -0.13542E 01 0.60536E 01 
0.4200E 00 0.64928E 00 0.37513E 01 0.14066E 01 0.60698E 01 
0.4400E 00 0.57264E 00 0.48423E 01 0.10244E 01 0.60967E 01 
0.4600E 00 0.49196E 00 0.61614E 01 0.64057E 00 0.61169E 01 
0.4800E 00 0.40905E 00 0.77644E 01 0.25639E 00 0.61040E 01 
0.5000E 00 0.32679E 00 0.97147E 01 -0.12523E 00 0.60316E 01 
0.5200E 00 0.24884E 00 0.12081E 02 -0.49999E 00 0.58849E 01 
0.5400E 00 0.17886E 00 0.14950E 02 -0.86327E 00 0.56684E 01 
0.5600E 00 0.11943E 00 0.18458E 02 -0.12113E 01 0.54035E 01 
0.5800E 00 0.71615E-01 0.22900E 02 -0.15419E 01 0.51179E 01 
0.6000E 00 0.35037E-01 0.29109E 02 0.12871E 01 0.48360E 01 
0.6200E 00 0.84207E-02 0.41493E 02 0.99160E 00 0.45738E 01 
0.6400E 00 0.98863E-02 0.40099E 02 0.71175E 00 0.43391E 01 
0.6600E 00 0.21544E-01 0.33334E 02 0.44571E 00 0.41340E 01 
0.6800E 00 0.28034E-01 0.31046E 02 0.19166E 00 0.39574E 01 
0.7000E 00 0.30599E-01 0.30286E 02 -0.52120E-01 0.38064E 01 
0.7200E 00 0.30248E-01 0.30386E 02 -0.28713E 00 0.36778E 01 
0.7400E 00 0.27784E-01 0.31124E 02 -0.51469E 00 0.35685E 01 
0.7600E 00 0.23845E-01 0.32452E 02 -0.73591E 00 0.34756E 01 
0.7800E 00 0.18939E-01 0.34453E 02 -0.95174E 00 0.33967E 01 
0.8000E 00 0.13470E-01 0.37413E 02 -0.11630E 01 0.33297E 01 
0.8200E 00 0.77640E-02 0.42198E 02 -0.13704E 01 0.32730E 01 
0.8400E 00 0.20841E-02 0.53622E 02 0.15671E 01 0.32252E 01 
0.8600E 00 0.33559E-02 0.49484E 02 0.13658E 01 0.31852E 01 
0.8800E 00 0.83817E-02 0.41533E 02 0.11667E 01 0.31520E 01 
0.9000E 00 0.12852E-01 0.37820E 02 0.96953E 00 0.31250E 01 
0.9200E 00 0.16655E-01 0.35569E 02 0.77388E 00 0.31037E 01 
0.9400E 00 0.19701E-01 0.34110E 02 0.57941E 00 0.30874E 01 
0.9600E 00 0.21922E-01 0.33182E 02 0.38580E 00 0.30761E 01 
0.9800E 00 0.23274E-01 0.32663E 02 0.19276E 00 0.30693E 01 
0.1000E 01 0.23727E-01 0.32495E 02 0 .60857E-08 0.30671E 01 
COEFFICIENTS OF DIGITAL FILTER WITH SPEC. AMPLITUDE AND/OR GROUP DELAY 
= 5 
0.2977922E-02 5 3 
0.1000000E 01-0.4252608E 01 0.1000000E 01-0.1035932E 01 0.2977080E 00 
0.1000000E 01 0.7839446E 00 0.1000000E 01-0.6341478E 00 0.3229899E 00 
0.1000000E 01 0.1774889E 01 0.1000000E 01 0.1860976E-01 0.4137183E 00 
0.1000000E 01-0.5258043E 00 0.8778015E-01 0. 0. 
0.1000000E 01-0.5990013E 01 0.1139210E 02 0. 0 
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An Optimization Program for the 
Design of Digital Filter Transfer Functions 


M. T. Dolanand J. F. Kaiser 


Digital Systems Research Dept. 
Bell Laboratories 
Murray Hill, NJ 07974 


1. Purpose 


Given the cascade realization of a digital filter transfer function, this program varies the coefficents 
until the performance meets arbitrary frequency-domain specifications on the magnitude. The response 
is calculated as insertion loss in dB. 


2. Method 


The method solves the design problem by working directly with the coefficients of the transfer func- 
tion 
A yz. + Qy,z! + ] 


H(z" ;a) =a 
: II a3,2 7 +04,27' +1 


(1) 


where @ = [ag, ay), &21, @31, 041, .. . ,@4,]4 z7! = e-7 and T is the sampling period. Note that N=4n 
is the total number of coefficients in n second-order sections. For a given sampling rate, the system is 
completely defined if » and @ are known. This initial design is usually obtained by standard approxima- 
tion techniques [1]. 


Let L(w;a@) be a function of the filter on which insertion loss specifications are imposed and 
I,\(w), I(w) be two given real-valued functions of w defined, respectively, on ©, and 5, two not- 
necessarily-disjoint intervals on the frequency axis. Then, all the common filter or equalizer 
specifications can be expressed by requiring the following inequalities to be satisfied: 


f\(@;a@) = L (w;a) a |, (w) >0, wc, (2) 


fy(w3a) = b(w) — L (wa) 50, et 0, 
A vector @ of a given dimension which satisfies the inequalities belongs to the specification set S, 


A stable digital filter for which aCS will be called acceptable, the philosophy being that a solution (not 
necessarily a global optimum) satisfies the specifications. There are no constraints on Stability. Since 
we are dealing with insertion loss only, if a final design is unstable, it is made stable by inversion of the 
poles which lie inside the unit circle in the z~! plane. 


The specification set S is defined in (3) through an infinite number of constraints. Because of the 
discrete nature of the computation, the user must select a finite number of discrete frequencies w, in 
; and ],. The program works with the discretized specification set 


Sy = lalf(o,;a) = L(@,;a) — I\(w,) 20, CO), k=1,2,...,my; (4) 


Figure | is an example of a lowpass filter H(z~! ;a) with frequency response L (w,;a). The filled 
circles mark the selected frequencies w,. The vector a belongs to the discretized specification set S, of 
(4) which in this case is a subset of S since the specifications are not violated between the selected fre- 
quencies. Note that S, is, of course, not necessarily a subset of S because there may exist a C S, and 
w C 2, ©) which do not satisfy the original specifications (2). This makes the selection of the 
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INSERTION LOSS IN dB. 


Fy Fo Fs 
FREQUENCY IN Hz 


Fig. 1 Lowpass filter response meeting discretized specifications 
e — selected frequencies ox, 


01: o%, 0K m < A, k =1,2,...,m and fy < om < fy, k = mti,...,.m 
Q,:0,,0K qy <i,k=mtl Seca m 


frequencies w, somewhat of an art because one would like to keep their number, m, as low as possible 
(in order to economize computation) but, at the same time, consider enough constraints to have a rea- 
sonable assurance that the solution a of an optimization problem based on S, will also belong to S. 


The optimality criterion adopted is of the following min-max character: If a filter of a given order is 
acceptable, then maximize the minimum amount by which it exceeds the specifications, if it is not 
acceptable, then minimize the maximum amount by which it fails. 


The problem is expressed mathematically by making use of an additional variable ay, (excursion 
from specifications) and introducing non-negative weights w,, and w», multiplying ay+; in each con- 
straint. The problem becomes: 
minimize 

AN+1 


subject to 
Ci, (a) = Wk AN+4+1 + fi (a, 3a) 20, k=1,2,...,m (5) 


Crx (a)= W2, AN+1 + fy (wy, 3a) 20, k=m,+1,m,+2,...,m 


The optimization is’ performed in the space of the augmented vector @ = [alay4,]'. Clearly, if ays, 
the final value of ay), is nonpositive, then the filter meets the discretized specifications. If ay+, > 0, 
then the final filter is not acceptable and ay,, represents the minimum of the maximum amount by 
which the constraints of S,; in (4) cannot be satisfied. Similar formulations have been used in approxi- 
mation methods [2], in the parameter optimization of networks in the time [3] or frequency domain 


[4,5,6], and in the design of antennas [7]. 


The values of the input weights may be adjusted by the user so that certain regions of ©, or 0) 
require greater attention than others. 


The solution of the nonlinear program of (5) by a sequence of unconstrained minimizations is 
accomplished through the use of an auxiliary or "penalty" function A(@;r,), where r, is a positive 
parameter. The auxiliary function A (@;r,) consists of the objective function plus a penalty term which 
is a monotonic function of the constraints multiplied by r, and so formulated that it reduces the value 
of A(a@;r,) when & belongs to S, and/or "penalizes" the minimization of A(a@;r,) when @ is on the 
boundary or the exterior of the specification set Sj. A(&;r,) is minimized sequentially over all 
& C Rt! for a sequence of r,, v = 1,2.,..., giving minima & (r,). If A(&;r,) and the sequence <r, > 


~~ 


are properly constructed, then the sequence <& (r,)> converges to the solution & of the original 
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nonlinear program [8]. In all cases, as the auxiliary function is minimized for a sequence of values r,, 
the penalty term tends to zero so that the successive minima of the auxiliary function converge to the 
minimum of the objective function. 


Two auxiliary functions used in the program are the following: 


a I ul l 
A, (@;r,=a Fs. eemeremra eS eines (6) 
2 Ci, (a) a C4 (@) 
sk m 
Aa;r,)= ai55 2 F, >> In(C,,(@)) ee = In(C}, (@)) (7) 
bel 


When either of these functions is minimized sequentially, for a monotonically decreasing sequence of 
positive r, (which tend to 0), then the sequence of the successive global minima &'(r,) converges to 
the solution of (5). A detailed treatment of the properties of these techniques is given in chapter 3 of 
reference [8]. 


Functions (6) and (7) are often called "interior" because they have the property that, if the initial 
value of @ belongs to S,, then all the subsequently generated solutions &‘(r,) are in S, as well. Both 
(6) and (7) approach infinity as any constraint approaches zero; therefore, any minimization procedure 
will tend to avoid the boundary of the constraint set. In interior methods, each time r, is reduced, the 
weighting of the penalty term is decreased while the weighting of the objective function is increased. As 
for the initial value of @ belonging to Sj, one can always find such an initial point by taking the initial 
value of ay4+, large enough. When an interior penalty function is selected, the program automatically 
determines this initial value for the user. 


Auxiliary functions which can work with points outside of . are called "exterior". The program 
incorporates one such function as an option: 


my 


A; (Gr) ey tr, 1C,@P +r, ¥ 16,0 (8) 
k=] k=m,+1 
where 
C,,(&@)= 0 if C,,(a@) >0 


a Ci, (a) if Ci, (@) =< 


and similarly for Cte. This function is minimized sequentially for a monotonically increasing 
sequence of positive r, (which tend to o) and has analogous convergence properties with those of (6) 
or (7). See [9] or Chapter 4 of [8]. In exterior methods, the penalty term prevents the points from 
straying too far from the feasible region. Only the constraints (5) which are violated enter the compu- 
tation at any time. It is especially true in exterior methods that the sign of ay4; is not significant 
before convergence. Convergence implies that subsequent values of r, will reduce ay, by amounts 
too small to justify the expenditure. In a wide range of problems, four values of r,, 1.e., four major 
iterations have been sufficient. Typically, in exterior methods, the initial r, is 1 and is multiplied by 10 
in each succeeding major iteration. In interior methods, the initial r, 1S usually less than 1 and is 
divided by 10 in succeeding major iterations. 


The test program of the Appendix provides an option for automatic selection of many of the optimi- 
zation parameters for both types of auxiliary functions. 


The unconstrained minimizations of the above auxiliary functions begin with an initial point @, and 
generate a sequence of points <@; >, such that 


A (@j4137,) < AQ; 5r,) (9) 
by taking a step o; along a direction vector d, C R%*!. Thus, 
Qi4) = &, + 0,0, (10) 


A class of unconstrained minimization methods, known as quasi-Newton methods, compute the direc- 
tion vectors as 
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d= — O8; (11) 


where g, = g(&;) is the gradient vector of A (a:r,) with respect to @ and Q; is a positive-definite 
(N+1)x(N+1) matrix which is updated at each iteration. Broyden [10] suggested the following general 
updating expressions 


0.4, = O; — O,yipi + Bia (12) 

where 
Yi = Siti — 8) (13) 
B; = Oi+1 ae a; (14) 


and the vectors p,; and q; are arbitrary except for the normalizing condition 
Piyi = Gy; = | | (15) 


The specialization of (12) included in the program is the well-known method of Davidon-Fletcher- 
Powell [11]. 


i am (y/O;y))~' Oly; (16a) 
q; = (Bly) 'B; (16b) 


The length of the step a, is determined at each iteration by minimizing A (a@;r,) along d; by means 
of a quadratic approximation of the function along dj. 


Finally, there is a choice of algorithms for "stop" criteria. Note that convergence to a negative a4; 
indicates that the frequency response satisfies the specifications at the frequencies chosen by the user. 
Users should check the resulting transfer function by generating a frequency response using the subrou- 
tine CASC of the Appendix. Comments in the subroutine include notes for its use outside the pack- 
age. If specifications are violated between the frequencies selected, additional frequencies should be 
included. The optimization is greatly facilitated by a good initial design and a judicious choice of fre- 
quencies at which the specifications must be met. 


3. Program Description 


3.1 Usage 


The package consists of a test program and nine subroutines - ZR, FLPWL, STEP, QUAD, FUNC, 
FG, CASC, RINV, FINISH. The test program and the subroutines are in the Appendix. 


The test program reads in the data required to describe the digital filter transfer function and the 
frequency-domain specifications calculated as insertion loss in dB. Various optimization parameters 
must also be read in. The coefficients are varied by the appropriate subroutines until the "stop" cri- 
terion is satisfied. 


For the sake of portability, all data is read in as formatted input. In some situations such as in the 
case of linear frequencies or analytic specifications, users may want to modify the test program to gen- 
erate such data or read it from files. 


ZR calculates the initial ay,,; (ALPHA) and sets the initial value r, (R) for the chosen penalty 
function (6), (7), or (8). Users have the option to set the initial r,,. 


FLPWL computes the direction vector (11) for each iteration. 

STEP sets the initial value of 7; of (10). 

QUAD performs on the function to be minimized quadratic interpolations for successive minima. 
FUNC _ evaluates the penalty function. 

FG evaluates the penalty function and its gradient. 


CASC calculates the frequency response and its gradient. The response is calculated as insertion loss 
in dB. 


RINV checks that the modified transfer function printed after each major iteration is stable. Since 
the coefficients are in the z~! plane, poles are inverted if necessary so that they lie outside the 
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FINISH 


OPTIIR 


unit circle. 


checks whether or not the stop criterion has been met and takes the appropriate action. 


3.2 Description of Required Data 


The notation below is not quite consistent with that in the earlier general discussion; it is instead 
consistent with the actual code and printout. 


NORD 


XK 


NC 


NL 


NG 


NF 


FREQ 
specifications at these selected frequencies. 


N1 


SPEC1 


TH1 


N2 


| (ORDER) The order of the filter must be an even number, NORD< 20 (NORD=2n where 


n is the number of second-order sections). For odd order filters, use the next higher even 
order, set appropriate coefficients to zero and request that they remain constant. For exam- 
ple, if the numerator and denominator are both to be seventh order, set A2, and B2, equal 
to zero in the array of coefficients described below. 


(SAMPLING TIME IN SECONDS) T = 1/sampling-frequency. 


(GAIN FACTOR) On input, XK contains the gain factor associated with the initial design. 
On output, it is the gain factor associated with the final design. 
(ARRAY of COEFFICIENTS) On input, X contains the coefficients as the transfer function 
of the initial design. For example, let 
A2,z~-?+A1,27!+1 
H ee ee eee 
(2 sx . . I B2 jz °+B1 2 til 


The system expects the number of coefficients to be twice the order of the filter and in the 
following sequence: 


a= [A2,,41,,B2,,B1,,A 25,4 15,B2),B 13... i & Ser. 2 ¥ a: x Ao we 
On output, X contains the coefficients of the optimized transfer function. 


(NUMBER OF COEFFICIENTS TO REMAIN CONSTANT) Any of the 4x coefficients may 
be held fixed during the optimization. NC = total count of those coefficients that must not 
change. 


(ARRAY OF INDICES OF FIXED COEFFICIENTS) NL contains the indices of coefficients 
to be held fixed. In the coefficient vector, x, index 1 refers to 42,, 2 refers to A 1, etc. 


(GAIN OPTION) If NG = 1, the optimizer changes the initial gain factor in addition to the 
coefficients free to change. If NG = 0, the gain remains unchanged. In most cases, it is 
advisable that the gain factor be free to vary. 


(NUMBER OF FREQUENCIES FOR OPTIMIZATION) NF = 
available to the optimizer - limit is 42. 


(ARRAY OF FREQUENCIES IN HZ) The goal of the optimizer is to satisfy the 


count of the frequencies 


(INDEX FOR TOP RANGE (1 - N1) OF LOWER SPECIFICATIONS) If, for example, the 
lower specifications apply to the lst through 10th frequencies, the user would input 10 for 
Nl. The 1 at the beginning is fixed. This is not a serious restriction since frequencies do not 
have to be in any special order. The first Nl can be chosen from anywhere in the range. Of 
course, the desired order must be set up in the frequency array. Note that the goal of the 
optimizer is to keep the loss above (greater than) the lower specifications at the given fre- 
quencies. 


(ARRAY OF LOWER SPECIFICATION DATA) Contains lower specifications calculated as 
loss in dB. 


(ARRAY OF WEIGHTS FOR LOWER SPECIFICATIONS) Contains non- are ey weights 
associated with the lower specifications. 


(LOWER INDEX FOR RANGE (N2 - N3) OF UPPER SPECIFICATIONS) If, for example, 
the upper specifications apply at the 10th through 20th frequencies, the user inputs N2 equal 
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N3 


SPEC2 
TH2 


NGID 


NOUT 


NN 


VBND 


NPEN 


to 10. 


(UPPER INDEX FOR RANGE (N2 - N3) OF UPPER SPECIFICATIONS) When the upper 
specifications apply at the 10th through 20th frequencies, e.g., the user inputs N3 equal to 20. 
Note that the optimizer works to keep the loss below (less than) the upper specifications at 
the given frequencies. 


(ARRAY OF UPPER SPECIFICATION DATA) Contains the upper specification data calcu- 
lated as loss in dB. 


(ARRAY OF WEIGHTS FOR UPPER SPECIFICATIONS) Contains non-negative weights 
associated with the upper specifications. 


(OPTIMIZATION GUIDE OPTION) Set NGID=1 and no additional input is necessary. The 
remaining parameters will be set by the test program. Set NGID=O to guide the optimiza- 
tion, i.e., to choose values for the remaining parameters. 


(OUTPUT OPTION) NOUT=0 for final results of major iterations, NOUT=1 for detailed 
output of minor iterations. Along with the initial coefficients and the R value, final results 
include the final coefficients for each major iteration, the gain factor, maximum excursion 
from specifications (ALPHA), penalty function value, modified coefficients (poles in ae 
plane made to lie outside unit circle), corresponding modified gain factor, the gradient vector, 
the number of function evaluations, and the number of gradient evaluations. Detailed output 
includes, of course, the results described above and in addition, for each minor iteration, the 
coefficient vector, gain factor, maximum excursion from specifications (ALPHA), penalty 
function value, initial step, and the number of times the constraints were violated during qua- 
dratic fit. Note that the number of minor iterations corresponds to the number of gradient 
evaluations. On rare occasions, it may be of interest to follow the optimization, but the user 
is warned that NOUT=1 often produces a wallpaper effect. 


(INITIAL STEP OPTION - NN=1 FOR STEP BASED ON MAX. COEF. EXPECTED; 2 
FOR STEP BASED ON APPROX. FUNC. MIN.; 3 FOR OPTION 1 ONCE THEN DIVI- 
SION BY 2) For option 1 user is expected to provide an estimate of the maximum coefficient 
expected (VBND). 


VBND 


So, @6t) 2 Se 
max. abs. component of direction vector 


For option 2 user is expected to provide an estimate of the function minimum (VBND). 


—2 x (func. — VBND) 


STEP=min SS ee ee 
gradient transpose X direction vector 


2, 


Since option 3 uses option 1 once, user is expected to provide VBND. Subsequently, the 
current step will be divided by 2 for a new step. 


(PARAMETER FOR INITIAL STEP OPTION) For step options 1 or 3, VBND is the max- 
imum coefficient expected. If user inputs 0., default option sets VBND equal to 
05 xX (initial maximum absolute coefficient). For step option 2, VBND is the approximate 
function minimum. There is no default option. | 


(PENALTY FUNCTION - NPEN=1 FOR RECIPROCAL INTERIOR; 2 FOR LOGA- 
RITHMIC INTERIOR; 3 FOR PARAMETRIC EXTERIOR) Penalty Function: 


P=atr, ole(x)] 


where a = the maximum excursion from specifications and r, is the positive appt of 
(6), (7), and (8) of the general discussion. 


Option | sets 


re N3 | 
ole ey 4" Fi 
kel 1k k=N2 ©2k 


Section 6.3 


OPTIIR 


Option 2 sets 


NI N3 
dle(x)] = — bs infe;,) + ¥ in(cp,) 
k=l k 


‘=N2 


Option 3 sets 


Nl 9 N3 Fe 
dle(x)] = > Cik KS > Crp 
k=] A=N2 


6 ta, eu 


where Cit — 4 if Cts a 0 


and similarly for ¢),. 


The constraint set c(x) is defined as follows 


where 


KP 


EPS 


KTIME 


NFINR 


p= Li) + wie h, SO «ook =, 2530.1 
Cy, =-L(f,) twyath, >0  k =N2,N241,....N3: 


L(f,) is the filter loss in dB at the k'" frequency. 

Wie is the weight associated with the lower specification at the k' frequency. 
= is the lower specification at the k' frequency. 

W, is the weight associated with the upper specification at the k'® frequency. 
bs is the upper specification at the k' frequency. 


a is the maximum excursion from the specifications. 


(STOP CRITERION FOR MINOR ITERATIONS. KP=1 FOR STOP BASED ON GRA- 


DIENT CHANGES; 2 FOR STOP BASED ON FUNCTION CHANGES; 3 FOR OPTION 2 
SATISFIED A GIVEN NO. OF TIMES; 4 FOR STOP BASED ON COEFFICIENT 
CHANGES; 5 FOR STOP AFTER FIXED NO. OF ITERATIONS) Option 1 - stop when 
gradient transpose times gradient is less than some value given by user. Option 2 - stop 
when present function value minus previous function value is less than .0001 times present 
value. Option 3 - stop when option 2 is satisfied a number of times set by the user. Option 
4 - stop when maximum absolute coefficient minus previous maximum absolute coefficient 
is less than a given value set by the user. Option 5 - stop after a given number of iterations 
set by user. 


(EPS - PARAMETER FOR MINOR STOP OPTION) This parameter is used in stop 
options | and 4. The default value for option 1 is gradient-transpose times gradient divided 
by 1.E+06. Default value for option 4 is .0005 times present maximum absolute coefficient. 
Input 0. for default. 


(PARAMETER FOR MINOR STOP OPTIONS) This parameter is used in stop options 3 
and 5. For option 3 the default number is 4. 


(STOP CRITERION FOR MAJOR ITERATIONS. NFINR=1 FOR STOP BASED ON 
ALPHA CHANGES OR SIZE OF CONSTRAINTS-SQUARED; 2 FOR STOP AFTER 
FIXED NO. OF R-VALUES) Option 1 - for interior reciprocal and logarithmic penalty func- 
tions - stop when present ALPHA value minus previous ALPHA value is less than a given 
percentage of the present value. User provides percentage. Option 1 - for parametric exte- 
rior penalty function - Program stops when sum of constraints-squared is less than a given 
value. User provides the value. Option 2 - may be chosen for any penalty function - User 
provides the number of major iterations desired. 
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PERR (PERCENTAGE) This parameter is used for stop option 1 with interior penalty functions. 


TAZ (SUM OF CONSTRAINTS-SQUARED) Parameter used only with stop option | with exte- 
rior penalty function. 
NR (NUMBER OF R-VALUES) User is expected to provide this number for stop option 2. 


INIR (R GUIDE OPTION - INPUT 0 FOR AN AUTOMATIC INITIAL R-VALUE AND 
CHANGE FACTOR, OTHERWISE INPUT 1) 
If the user inputs 0, a good starting r-value will be calculated based on the assumption that 
the two terms in the penalty function be of the same order of magnitude. Also the change 
factor is set to 10. 


RINI (INITIAL R-VALUE) Used only if the user set INIR=1. 
RCHNG (R CHANGE FACTOR) Used only if user set INIR=1. 


4. Test Problem 


Design a lowpass filter with cutoff frequency 4000 Hz for a sampling frequency of 32000 Hz. The 
following limits are imposed on the magnitude-frequency characteristic: 


LOWER SPECIFICATIONS 
In the passband, the insertion loss is required to be greater than 


2 
-01 + 20.xlogo| Sminslaono) |g 


In the stopband, the insertion loss is required to be greater than 


sin (zr. f/8000) IB 


30,-+ 20. Xlogio 7 f/8000 


UPPER SPECIFICATIONS 
In the passband, the insertion loss is required to be less than 


2 
+0.1. + 20.xI0819 eae ate | d 


For an initial design, the choice is a lowpass sixth-order elliptic filter with a minimum stopband loss 
of 30 dB and aripple factor of 0.2 dB. 


The main program (Appendix) reads in the parameters described in Section 3.2 and for each major 
iteration prints, among other things, the transfer function, the gain factor, and the maximum excursion 
from the specifications, ALPHA. Convergence to a negative ALPHA means that the filter meets the 
specifications at the frequencies selected. 


Table 1 contains the input data and Table 2 the output. Figure 2 shows the frequency response for 
the 3 


INITIAL 
60.0 / RESPONSE 


INSERTION 
LOSS IN dB 
) 
O 
oO 


O 2000 4000 6000 8000 10000 12000 14000 16000 
FREQUENCY IN HZ 


Fig. 2 Example - initial design 


initial design. Filled circles mark the discrete frequencies w, of (4). Note that the specifications are 
satisfied in the stopband but violated in the passband. Figure 3 is a blowup of the passband region of 
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INITIAL RESPONSE 


UPPER SPECS 


LOWER SPECS—_\ 


INSERTION LOSS IN dB 


SPEC PONT ———" 


0 800 1600 2400 3200 4000 
FREQUENCY IN HZ 


Fig. 3 Blowup of passband - initial design 
the initial design showing that the specifications are violated by approximately 5 dB. 


Figure 4 is the optimized design. The stopband underwent little 


80.0 

co FINAL 
60 600r UPPER SPECS | RESPONSE 
EZ 400+ j ower 
“Wi 200+ SPECS 
> YY) 
= So O 


edt 
O 2000 4000 6000 8000 10000 12000 14000 16000 
FREQUENCY IN HZ 
Fig. 4 Example - final design 


change during the optimization. Figure 5 is a blowup of the passband region of the optimized 


UPPER SPECS 


\ FINAL RESPONSE 


INSERTION LOSS IN dB 


e) 800 1600 2400 3200 4000 
FREQUENCY IN Hz 


Fig. 5 Blowup of passband - final design 


design where most of the change took place. The specifications are clearly satisfied. 
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95917120E+00 -0.13700062E+01 


Table 1 

6 0.31250000E-04 0.48525318E-01 
0.10000000E+01 -0.13200034E+01 0. 
0.10000000E+01 -0.11366976E+01 0.79029629E+00 
0.10000000E+01 0.30500371E+00 0.43064909E+00 
3 

— 

1 28 

0.10000000E-02 0.39979457E+03 0.69890112E+03 
0.12930097E+04 0.15870320E+04 0.18784126E+04 
0.24515404F+04 0.27325081E+04 0.29175082E+04 
0.31913099E+04 0.44034865E+04 0.46480381E+04 
0.54222495E+04 0.61329350E+04 0.67815157E+04 
0.79061884E+04 0.88310266E+04 0.95931836E+04 
0.10754695E+05 0.11201857E+05 0.12446131E+05 
28 
-0.10000000E+00 -0.17149741E+00 -0.31933432E+00 
-0.86123592E+00 -0.12584475E+01 -0.17428471E+01 
-0.29861407E+01 -0.37534815E+01 -0.43226320E+01 
-0.52677316E+01 0.18688911E+02 0.16903418E+02 
0.90909537E+01 -0.42527329E+01 -0.27946299F+03 
0.21700557E+00 0.30703544E+01 -0.56281188E+01 
-0.99938936E+01 -0.58035957E+01 -0.11648717E+02 
0.10000000E+01 0.10000000E+01 0.10000000E+01 
0.10000000E+01 0.10000000E+01 0.10000000EF+01 
0.10000000E+01 0.10000000E+01 0.10000000E+01 
0.10000000E+01 0.10000000E+01 0.10000000E+01 
0.10000000E+01 0.10000000E+01 0.10000000E+01 
0.10000000E+01 0.10000000E+01 0.10000000E+01 
0.10000000E+01 0.10000000E+01 0.10000000E+01 
1 13 

0.10000000E+00 0.28502590E-01 -0.11933433E+00 
-0.66123592E+00 -0.10584475E+01 -0.15428471E+01 
-~0.27861407E+01 -0.35534815E+01 -0.41226320E+01 
-0.50677316E+01 

0.10000000E+01 0.10000000E+01 0.10000000E+01 
0.10000000E+01 0.10000000E+01 0.10000000E+01 
0.10000000E+01 0.10000000E+01 0.10000000E+01 
0.10000000E+01 

0 

Table 2 


INITIAL COEFFICIENTS FOR R= 
A2 A1 
0.10000000E 01 -0.13200034E 01 
0.10000000E 01 -0.11366976E 01 
0.10000000E 01 0.30500371E 00 


0.95917120E 00 


0.43064909E 00 


0.26582312E-03 


B2 


=0 
0 


GAIN= 0.48525318E-01 ALPHA= 0.51585356E 01 FUNCTION= 


FINAL COEFFICIENTS FOR R= 
A2 A1 
0.10000000E 01 -0.11005577E 01 
0.10000000E 01 -0.12672789E 01 
0.10000000E 01 0.44910363E 00 


0.86160736E 00 


0.42624463E 00 


0 .26582312E-03 


B2 


GAIN= 0.51341804E-01 ALPHA=-0.15522700E-01 FUNCTION= 


MODIFIED COEFFICIENTS 
0.10000000E 01 -0.11005577E 01 
0.10000000E 01 -0.12672789E 01 
0.10000000E 01 0.44910363E 00 

MODIFIED GAIN= 0.51341804E-01 


0.86160736E 
0.74878288E 
0.42624463E 


00 


-13095079E+01 


-11934469E+01 


- 99680567E+03 


-21667174E+04 


-31005501E+04 


-50430803E+04 


.73711900E+04 


-10225487E+05 


«b3191388E+05 


.54880904E+00 


-23174821E+01 


-49401417E+01 


-13416951E+02 


.86185069E+01 


-27946299E+03 


-16014496E+02 


-10000000E+01 


. 1O000000E+01 


-10000000E+01 


- 10000000E+01 


-10000000E+01 


- 10000000E+01 


-10000000E+01 


. 34880904E+00 


-21174821E+01 


-47401417E+01 


- 10000000E+01 


-10000000E+01 


. 10000000E+01 


B1 


-0.13700062E 01 
0.79029629E 00 -0.13095079E 01 
-0.11934469E 01 
O<TOSETOFIE 02 


B1 


=O .13928926E 61 
0.74878288E 00 -0.12533824E 01 
-0.10688564E 01 
0.67633536E-01 


00 -0.13928926E 01 
-0.12533824E 01 
00 -0.10688564E 01 


GRADIENTS 
0. -0.15056353E 00 0.27614075E 00 0.33629320E 00 
0... -0.19561886E 00 0.22825324E 00 0.28943125E 00 
0 -0.48712567E-01 0.15260250E 00 0.21883522E 00 
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Table 2 
(Continued) 


INITIAL COEFFICIENTS FOR R= 0.26582312E-04 
A2 A1 B2 B1 
0.10000000E 01 -0.11005577E 01 0.86160736E 00 -0.13928926E 01 
0.10000000E 01 -0.12672789E 01 0.74878288E 00 -0.12533824E 01 
0.10000000E 01 0.44910363E 00 0.42624463E 00 -0.10688564E 01 
GAIN= 0.51341804E-01 ALPHA=-0.15522700E-01 FUNCTION=-0.72070768E-02 


FINAL COEFFICIENTS FOR R= 0.26582312E-04 
A2 A1 B2 B1 
0.10000000E 01 -0.10786655E 01 0.87831046E 00 -0.13160157E 01 
0.10000000E 01 -0.12562839E 01 0.78747962E 00 -0.12743440E 01 
0.10000000E 01 0.48262715E 00 0.44985502E 00 -0.10219863E 01 
GAIN= 0.72560707E-01 ALPHA=-0.73472910E-01 FUNCTION=-0.47247530E-01 


MODIFIED COEFFICIENTS 
0.10000000E 01 -0.10786655E 01 0.87831046E 00 -0.13160157E 01 
0.10000000E 01 -0.12562839E 01 0.78747962E 00 -0.12743440E 01 
0.10000000E 01 0.48262715E 00 0.44985502E 00 -0.10219863E 01 
MODIFIED GAIN= 0.72560707E-01 


GRADIENTS 
iz -0.35273176E 00 0.53998295E 00 0.72532947E 00 
0. -0.48105144E 00 0.44776873E 00 0.69719212E 00 
0. -0.10700078E 00 0.21568865E 00 0.44558926E 00 
198 FUNCTION EVALUATIONS 32 GRADIENT EVALUATIONS 


XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXKXXKXXKXXXXXXXXXXXXXKXXXK 


INITIAL COEFFICIENTS FOR R= 0.26582312E-05 
A2 A1 B2 B1 
0.10000000E 01 -0.10786655E 01 0.87831046E 00 -0.13160157E 01 
0.10000000EFE 01 -0.12562839E 01 0.78747962E 00 -0.12743440E 01 
0.10000000E 01 0.48262715E 00 0.44985502E 00 -0.10219863E 01 
GAIN= 0.72560707E-01 ALPHA=-0.73472910E-01 FUNCTION=-0.70850371E-01 


FINAL COEFFICIENTS FOR R= 0.26582312E-05 
A2 A1 B2 B1 
0.10000000E 01 -0.10733848E 01 0.87859791E 00 -0.13189261E 01 
0.10000000E 01 -0.12641738E 01 0.78693668E 00 -0.12579032E 01 
0.10000000E 01 0.48517703E 00 0.46097852E 00 -0.10228557E 01 
GAIN= 0.76552721E-01 ALPHA=-0.91377584E-01 FUNCTION=-0.83240161E-01 


MODIFIED COEFFICIENTS 
0.10000000E 01 -0.10733848E 01 0.87859791E 00 -0.13189261E 01 
0.10000000E 01 -0.12641738E 01 0.78693668E 00 -0.12579032E 01 
0.10000000E 01 0.48517703E 00 0.46097852E 00 -0.10228557E 01 
MODIFIED GAIN= 0.76552721E-01 


GRADIENTS 
0. -0.68659899E-01 0.80960995E-01 0.21131991E 00 
0. -0.12261476E 00 -0.66190683E-02 0.14751522E 00 
0. -0.11043458E-01 -0.15977386E 00 -0.45411769E-01 
89 FUNCTION EVALUATIONS 12 GRADIENT EVALUATIONS 
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INITIAL COEFFICIENTS FOR R= 0.26582312E-06 
A2 Al B2 B1 
0.10000000E 01 -0.10733848E 01 0.87859791E 00 -0.13189261E 01 
0.10000000E 01 -0.12641738E 01 0.78693668E 00 -0.12579032E 01 
0.10000000E 01 0.48517703E 00 0.46097852E 00 -0.10228557E 01 
GAIN= 0.76552721E-01 ALPHA=-0.91377584E-01 FUNCTION=-0.90563842E-01 


FINAL COEFFICIENTS FOR R= 0.26582312E-06 
A2 A1 B2 B1 
0.10000000E 01 -0.10733774E 01 0.87862835E 00 -0.13189427E 01 
0.10000000E 01 -0.12641572E 01 0.78697857E 00 -0.12578720E 01 
0.10000000E 01 0.48517863E 00 0.46098580E 00 -0.10228711E 01 
GAIN= 0.76564977E-01 ALPHA=-0.96739767E-01 FUNCTION=-0.94341949E-01 
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Table 2 


(Continued) 


MODIFIED COEFFICIENTS 
0.10000000E 01 -0.10733774E 01 
0.10000000E 01 -0.12641572E 01 
0.10000000E 01 0.48517863E 00 

MODIFIED GAIN= 0.76564977E-01 


GRADIENTS 
0. 0.16562078E 01 
0. 0.23699104E 01 
0 0.48901627E 00 


46 FUNCTION EVALUATIONS 


XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXKXXXXXXXXXKXXXXXXXKXX 


0.87862835E 00 -0.13189427E 
0.78697857E 00 -0.12578720E 
0.46098580E 00 -0.10228711E 


=0.258117159E 01 -0.35206183E 

-0 .20665732E 01 -0.32827339E 

-0.71702110E 00 -0.192639143E 
4 GRADIENT EVALUATIONS 
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A Program for Designing 
Finite Word-Length IIR Digital Filters 


Kenneth Steiglitz and Bruce D. Ladendorf 


Department of Electrical Engineering and Computer Science 
Princeton University 
Princeton, NJ 08540 


1. Purpose 


This program performs the design of finite word-length IIR digital filters to meet magnitude 
specifications in the frequency domain. 


2. Method 


This program is a somewhat generalized implementation of the algorithm described in [1]. The 
reader is referred to that paper for a detailed problem statement and description of the method. The 
program begins with a set of high precision coefficients for an IIR digital filter in cascade form. This 
filter has been designed by other means, and its magnitude transfer function meets tolerance require- 
ments in pass- and stopbands in the frequency domain. The problem is to design a new filter which 
still meets these requirements but has coefficients with no more than NBIT1 bits after the decimal 
point. The program obtains a starting point by rounding off the high precision coefficients to NBIT1 
bits, and then uses a randomized version of the Hooke and Jeeves search algorithm [2] to optimize the 
coefficients with an initial search increment of NBIT2 bits (NBIT2 < NBIT1). If the final function 
- value is no more than 1, the desired tolerances have been met. Because the search algorithm is ran- 
domized, different runs with different settings of the random number initialization, NSET, will in gen- 
eral give different answers, so that the best of several runs may be used. The input data is read in by 
SUBROUTINE INDATA; the heading of that subroutine describes the arrangement of the input data. 
The random number generator used is UNI (see Standards Section). 


3. Program Description 
The program is organized in the following subprograms: 
3.1 MAIN Program 


This is the executive program, which calls the others. The subprograms communicate largely 
through COMMON blocks ABC, RAW, and STEER. ABC has information about the coefficients X 
and other input parameters, such as the desired accuracy. RAW has information about the frequency 
domain specifications — a grid of frequency points is supplied, together with the corresponding desired 
values of the transfer function and tolerances. COMMON block STEER has three logical variables, 
PRINT, TWOPT, and FIXCOF, which determine whether the final results are printed by FUNCT, 
whether two-opt is performed, and whether the second numerator coefficients are fixed at 1, respec- 
tively. After optimization SUBROUTINE FUNCT is called with PRINT = .TRUE., which evaluates 
the transfer function on a finer grid and prints out the final results. 


3.2 SUBROUTINE INDATA 


This subroutine reads in all the problem data; the details are described in its heading. It shares this 
data through COMMON with the other programs. Note that the user should set NIWOPT = 1 to use 
two-opt, and 0 otherwise. 
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3.3 SUBROUTINE FUNCT 


This computes the value of the maximum weighted error function on the coarse grid, and at the 
end, on a finer grid. The constant multiplier of the filter, 4, is computed to high precision and not 
quantized. This calculation assumes that the desired transfer function values are 0 or 1; and that the 
weights in all the passbands are equal; otherwise this subroutine must be modified (see Section 4). 


3.4 SUBROUTINE HANDJ 


This is a randomized version of the Hooke and Jeeves Pattern Search. A block diagram is given in 


[1]. 
3.5 SUBROUTINE EXPLOR 


This is a local exploration program called by HANDJ; a block diagram is also given in [1]. The local 
exploration is controlled by the logical variables TWOPT and FIXCOF. If TWOPT is .TRUE. then the 
denser two-opt search is performed. If FIXCOF is .TRUE. then the second numerator coefficient in 
each section is frozen at the value 1. 


3.6 SUBROUTINE SET 
This is a program which simply sets one vector equal to another. 
3.7 SUBROUTINE SHUFF 
This program randomly orders the list of coordinates, LIST. It is used by EXPLOR. 


4. Calculations of the Gain Constant 4 


The gain constant A is kept to high precision and not optimized; it is recalculated on the basis of the 
coarse grid each time FUNCT is called, and at the end on the basis of a finer grid. It is important, 
therefore, to have an efficient way of obtaining A. In this implementation the following heuristic is 
used, which somewhat restricts the class of filters to which the program is applicable. The minimum 
and maximum values of the magnitude of the unnormalized transfer function N/D, Y, and Y, are cal- 
culated in the passbands only, and A is calculated by 


A = 2/(Y,+Y) 


This centers the magnitude characteristic in the passbands; assuming that the desired transfer function 
values are | in all the passbands (and 0 in all the stopbands), and that the tolerances in the passbands 
are all equal. It is heuristic even with these assumptions because it ignores the error in the stopbands. 
In most cases, however, there is little to be gained by de-centering the magnitude characteristic in the 
passband to reduce the error in the stopband. 


The exact calculation of A requires the solution of a linear program; at each point we write the con- 
straints 


AY — Yi << de 
~(AY—Y,) < de 


where Y is the magnitude of N/D, Y, is the desired magnitude characteristic, and d is the tolerance. 
The linear programming problem is to find A and e such that eis a minimum. 


5. Comments 


A. If NCOEF is specified as 4, 4 coefficients per stage are read in, and all of them optimized. If 
NCOEFF is specified as 3, it is assumed that the second numerator coefficient of each section is 
fixed at 1. This is often a reasonable assumption when the filter has stopbands, and has the 
advantages of a simpler filter implementation and an optimization problem with 3*NSECT instead 
of 4*NSECT parameters. 


B. The check mentioned in [1] that the first numerator coefficient be less than 2 in magnitude is 
omitted in FUNCT. Thus it is conceivable that the numerator coefficients become large in magni- 
tude, requiring more bits for their integer parts. 
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C. The grid is not generated automatically, but must be read in point by point. Experience has 
shown that it can have as few as 2.5*N points, where N is the number of adjustable parameters in 
the problem, provided that the points are chosen carefully. Since the running time of FUNCT is 
roughly proportional to the number of grid points, it is important to keep this number as small as 
possible. It has been found best to choose the points in clusters near the transition regions, and 
less densely elsewhere. 


D. The dominant computational burden is the DO 50 loop in FUNCT. This uses a formula close to 
that in [3] and requires 10 multiplication/additions per section per grid point, so the total number 
of such operations in this loop is 1O*NSECT*M*KOUNT, where NSECT = number of sections, 
M = number of grid points, and KOUNT = number of function evaluations. Thus, in the test 
problem, NSECT = 4, M = 28, and (for NSET = 100) KOUNT = 3181; so that there are 
3,562,720 multiplications/additions in the DO 50 loop. The total execution time for this problem 
(excluding compilation) was 4.26 seconds, using FORTRAN H on an IBM 360/91; 10.23 seconds 
using FORTRAN G on the same machine; and 38.8 seconds on the Honeywell 6080N. 


6. Test Problem 


A listing of the program is given in the Appendix. We use example | from [1], the bandpass exam- 
ple of Avenhaus and Schuessler [4], as a test example. The filter has the passband [0.411111, 
0.466666]; and stopbands [0.0, 0.383333] and [0.494444, 1.]. The test data is listed in Table 1, and the 
corresponding program output in Table 2. 


The first card (in FORMAT (513)) sets 


NCOEF = 3 
NSET = 100 
NTWOPT = 1 
NBIT1 = 6 
NBIT2 = 5 


The next 28 cards establish the grid and weighting, the next card says there are 4 sections, and the 
remaining 12 cards give the high precision coefficients. (See the heading of SUBROUTINE INDATA 
for detailed formatting information. ) 


With this value of NSET = 100, the local optimum is found after 3181 function calls, and 
corresponds to a value of F = 0.7681 on the fine grid. 
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Table 1 


WwW 
= 
Oo 
io) 
= 
fe) 
63] 


0. 0. 01 
0.36 0. 01 
0.3658 0. 01 
0.383333 0. 01 
ee (ee soe F 03 
0.412 = 03 
0.4129 +. 03 
0.414 a 03 
0.4157 1, 03 
0.4185 :: 03 
0.423 1 03 
0.4275 _ 03 
0.433 13 03 
0.4386 G 03 
0.4443 2 03 
0.45 He 03 
0.4545 1 03 
0.459 2 03 
0.461916 1. 03 
0.4636 .. 03 
0.464833 1. .03 
0.465749 1. 03 
0.466666 1. 03 
0.494444 0. 01 
0.51 0. 01 
0.54448 = 0. 01 
0.52 0. 01 
1 0. 01 

4 

0.4512591755000000 

-0 .2855823838000000 


0.9116860616000000 
-1.0986945504000000 
-4457342317000000 
0.9130786730000000 
.0099665157000000 
-1969016650000000 
0.9695353354000000 
.7304169706000000 
-5515388641000000 
0.9705899009000000 


Table 2 


*%*%*%** INPUT DATA «xxx 

NUMBER OF COEFFICIENTS PER STAGE IS 3 
RANDOM INITIALIZATION IS 100 

TwWOrT Is. -7 


FINAL ( AND INITIAL ROUNDING ) PRECISION IS 6 BITS 
INITIAL SEARCH DELTA IS_ 5 BITS 
GRID SPECIFICATIONS 
POINT NO. FREQUENCY DESIRED Y TOLERANCE 
1 Ds ie 0.1000000000D-01 
2 0.3600000000D 00 0. 0.1000000000D-01 
3 0.3658000000D 00 0. 0.1000000000D-01 
4 0.3833330016D 00 O. 0.1000000000D-01 
5 0.4111110016D 00 0.1000000000D 01 0.3000000000D-01 
6 0.4120000000D 00 0.1000000000D 01 0.3000000000D-01 
7 0.4129000000D 00 0.1000000000D 01 0.3000000000D-01 
8 0.4140000000D 00 0.1000000000D 01 0.3000000000D-01 
2 0.4157000000D 00 0.1000000000D 01 0.3000000000D-01 
10 0.4185000000D 00 0.1000000000D 01 0.3000000000D-01 
ra 0.4230000000D 00 0.1000000000D 01 0.3000000000D-01 
{zZ 0.4275000000D 00 0.1000000000D 01 0.3000000000D-01 
se 0.4330000000D 00 0.1000000000D 01 0.3000000000D-01 
14 0.4386000000D 00 0.1000000000D 01 0.3000000000D-01 
i 0.4443000000D 00 0.1000000000D 01 0.3000000000D-01 
16 0.4500000000D 00 0.1000000000D 01 0.3000000000D-01 
5 0.4545000000D 00 0.1000000000D 01 0.3000000000D-01 
18 0.4590000000D 00 0.1000000000D 01 0.3000000000D-01 
19 0.4619160000D 00 0.1000000000D 01 0.3000000000D-01 
20 0.4636000000D 00 0.1000000000D 01 0.3000000000D-01 


6.4-4 


Section 6.4 


Table 2 
(Continued) 

24 0.4648329984D 00 0.1000000000D 01 
22 0.4657489984D 00 0.1000000000D 01 
23 0 .4666660032D 00 0.1000000000D 01 
24 0.4944440000D 00 0. 

25 0.5100000000D 00 0. 
26 0.5144800000D 00 0. 
27 0.5200000064D 00 0. 
28 0.1000000000D 01 0. 


.3000000000D-01 
. 3000000000D-01 
.3000000000D-01 
- 1000000000D-01 
.1000000000D-01 
. 1000000000D-01 
-1000000000D-01 
.1000000000D-01 


© 'O'O'O'© © ' O'@ 


INITIAL HIGH PRECISION COEFFICIENTS FOR 4 SECTIONS ARE 


SECTION 1 
0.4512591744D 00 
-0.2855823808D 00 
SECTION 2 
-0.1098694544D 01 
-0 .4457342336D 00 
SECTION 3 
-0.9966515712D-02 
-0.1969016640D 00 
SECTION 4 
-0.7304169664D 00 
-0.5515388608D 00 


THE INITIAL COEFFICI 
SECTION 1 
0.4531249984D 00 
-0.2812500000D 00 
SECTION 2 
-0.1093750000D 01 
-0.4531249984D 00 
SECTION 3 
-0.1562500000D-01 
-0.2031250000D 00 
SECTION 4 
-0.7343750016D 00 
-0.5468750016D 00 


0.1000000000D 
0.9116860672D 


0.1000000000D 
0.9130786688D 


0.1000000000D 
0 .9695353344D 


0.1000000000D 
0.9705899008D 
ENTS, ROUNDED TO 


0.1000000000D 
0 .9062499968D 


0.1000000000D 
0 .9062499968D 


0.1000000000D 
0 .9687500032D 


0.1000000000D 
0 .9687500032D 


6 BITS, ARE 


NEXT FOLLOWS A REPORT FROM THE SEARCH ALGORITHM 


NO. CALLS PAT. SIZE _ DELTA OLD VALUE NEW VALUE 
264 1 0.31250000D-01 0..25259817D G4 0.17236617D 01 
805 1 0.31250000D-01 0.17236617D 01 0.17160663D 01 
1602 1 0.15625000D-01 0.17160663D 01 0.94060213D 00 
2126 1 0.15625000D-01 0.94060213D 00 0.73309429D 00 
2650 1 0.15625000D-01 0.73309429D 00 0.71935856D 00 
FINAL VALUES- NO. OF FUNCT CALLS= 3181 FUNCT= 0.7193585600D 00 
THE FINAL COEFFICIENTS, HAVING 6 BITS, ARE 
SECTION 1 
0.4375000000D 00 0.1000000000D 01 
-0.2812500000D 00 0.9062499968D 00 
SECTIGN. «2 
-0.1093750000D 01 0.1000000000D 01 
-0 .4531249984D 00 0.9062499968D 00 
SECTION 3 
-0.1562500000D-01 0.1000000000D 01 
-0.1875000000D 00 0.9687500032D 00 
SECTION 4 
-0.7343750016D 00 0.1000000000D 0Q1 
-0.5625000000D O00 0.9687500032D 00 
***** FINAL RESULTS ***** 


THE CONSTANT A IS 


0.7493357696D-02 


FUNCTION VALUE ON COARSE GRID IS 0.7193585600D 00 

FREQUENCY DESIRED Y ACTUAL Y ERROR 
GO. G.. 0.70283624D-02 0.70283624D 00 
0.36000000D 00 QO. 0.64137194D-02 0.64137194D 00 
0.36580000D 00 O. 0.67483316D-02 0.67483316D 00 
0.3833353S00D 00 0. 0..52989o 150-02. 0... 525895180 00 
0.41111100D 00 0.10000000D 01 0.10215808D 01 0.71935856D 00 
0.41200000D 00 0.10000000D 01 .0.10204763D 01 0.68254401D 00 
0.41290000D 00 0.10000000D 01 0.10148222D 01 0.49407221D 00 
0.41400000D 00 0.10000000D 01 0O.10066453D 01 0.22150962D 00 


FWIIR 


IIR Programs 


Table 2 
(Continued) 
0.41570000D 00 0.10000000D 01 0.99765911D 00 0 
0.41850000D 00 0.10000000D 01 0.99730467D 00 0 
0.42300000D 00 0.10000000D 01 0.10136004D 01 0O 
0.42750000D 00 0.10000000D 01 0.107 61582D 05°<90 
0.43300000D 00 0.10000000D 01 0.99671587D 00 0 
0.43860000D 00 0.10000000D 01 0O.98587546D 00 QO 
0.44430000D 00 0.10000000D 01 0.10004271D 01 £O 
0-45000000D 00 0.T0000000D 01- 0.109 54313001 <0 
0,45450000D-. 00. 0 TO0GGOCUD 01 0.10021 23 70: 04: 0 
0.45900000D 00 0.10000000D 01 0.97841925D 00 0 
0.46191600D 00 0.10000000D 01 0O.97883618D 00 0 
0.46360000D 00 0.10000000D:01 0.98927915D 00 0 
0.46483300D 00 0.10000000D 01 0.99955200D 00 0 
0.46574900D 00 0.10000000D 01 0.10052380D 01 0 
0.46666600D 00 0.10000000D 01 0O.10045265D 01 0O 
0.49444400D 00 O. 0.53608885D-02 0 
0.51000000D 00 OQ. 0.70073326D-02 0 
0.51448000D 00 0O. 0.71485085D-02 0 
0.52000000D 00 OO. 0.67435740D-02 0 
0.10000000D 01 O. 0.70871288D-02 0 
FINAL RESULTS ON FINE GRID- 10 POINTS PER GRID POINT 
FREQUENCY DESIRED Y ACTUAL Y 
*%*%*%*%* BAND FROM GRID POINT + Fo 2 KKK 
2 i 0.70355889D-02 0 
0.36000000D-01 QO. 0.70067577D-02 0 
0.72000000D-01 0. 0.69162904D-02 0 
0.10800000D 00 OO. 0.675171 103b-02 0 
0.14400000D 00 OO. 0.64849851D-02 0 
0.18000000D 00 OO. 0.60693616D-02 0 
0.21600000D 00 OO. 0.54133609D-02 0 
0.25200000D 00 0. 0.43383098D-02 0 
0.28800000D 00 0. 0.24698003D-02 0 
0.32400000D 00 OO. 0.98742474D-03 0 
***%*%* BAND FROM GRID POINT Zz 40 3 #H#HH 
0.36000000D 00 OO. 0.64203139D-02 0 
0.36058000D 00 OO. 0.64798328D-02 0 
0.361T6000D 00 VU. 0.65348297D-02 0 
0.36174000D 00 OQ. 0.65848927D-02 0 
0.36232000D 00 OO. 0.66295783D-02 0 
0.36290000D 00 O. 0.66684091D-02 0 
0.36348000D 00 QO. 0.67008707D-02 0 
0.36406000D 00 OO. 0.67264089D-02 0 
0.36464000D 00 OO. 0.67444262D-02 0 
0.36522000D 00 . G. 0.67542784D-02 0 
**%*%*%* BAND FROM GRID POINT Ss Fo 4 KKK 
0.36580000D 00 OQ. 0 .67552702D-02 .-0 
0.36755330D 00 =. 0.66964453D-02 0 
0.36930660D 00 0. 0.65250871D-02 0 
0. 373059900 G0. oo. 0.62103614D-02 0 
0..37281320D. 00 6. 0.57130142D-02 0 
0.37456650D 00 0. 0.49827854D-02 0 
0.37631980D 00 0. 0.39549038D-02 0 
0.37807310D 00 O. 0.25452870D-02 0 
0. 379826405 00° Gc. 0.64388579D-03 0 
6. s61Ss79700 UU 9. 0.18946638D-02 0 
0. 383333000 660 9; 0.52643590D-02 0 
***%*%* BAND FROM GRID POINT 5 3G G6 *#HK 
0.4111747100D 00. 0..10000CCOPD OT OG. td2260511D_0t 0 
0.41119990D 00 0.10000000D OT 0.10228557D 07- 0 
0.41128880D 00 0.10000000D 01 0.10229910D 01 O 
0.41137770D 00- 0.710000000D 01.. 0.10230429D OT 0 
0.41146660D 00 0.10000000D 01 0.10230170D 01 O 
0..47455550D U0 0.3 0p 0CCUULD OT Vie 2zetsoU Vt UD 
0.41164440D 00 0.10000000D 01 0.10227540D 01 0O 
O..41ti33300 U0 ©. 1eGgerorn C1 0.70225274D UT 0 
0.41182220D 00 0.10000000D 01 0.10222440D 01 0 
0.41191110D 00 0.T10000000D 01 0.10219085D 071 0O 
**%**%* BAND FROM GRID POINT 6 3G 7 KKK 
OG .41200000D GO © .TOGUG000UD U1 -0.1TO21S256D Ut 0 
0.41209000D 00 0.10000000D 01 0.10210939D 01 QO 
0.41218000D 00 0.10000000D 01 0.10206224D 01 .0 
0.41227000D 00 0.10000000D 01 0.10201150D 01 O 
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. 78030010D-01 
-89844010D-01 
-45334529D 00 
-53860584D 00 
-10947098D 00 
-47081791D 00 
-14235485D-01 
-51437758D 00 
. 701909 760-01 
.7 T835656D: 00 
.70546071D 00 
.35736149D 00 
. 14933567D-01 
.17460133D 00 
“15088357D. 00 
.53608885D 00 
.70073326D 00 
-71485085D 00 
.67435740D 00 
.70871288D 00 


ERROR 


.70355889D 00 
~ F0067577D 00 
-69162904D 00 
»67511103D °00 
-64849851D 00 
-60693616D 00 
-54133609D 00 
.43383098D 00 
-24698003D 00 
.98742474D-01 


.64203139D 00 
-64798328D 00 
.65348297D 00 
.65848927D 00 
-66295783D 00 
-66684091D 00 
.67008707D 00 
-67264089D 00 
.67444262D 00 
.67542784D 00 


»67552702D 00 
-66964453D 00 
65250871TD 00 
-62103614D 00 
-57130142D 00 
-49827854D 00 
.39549038D 00 
-25452870D 00 
.64388579D-01 
-18946638D 00 
-52643590D 00 


.75437147D 00 
-76185765D 00 
. 76636676D 00 
-76809520D 00 
.16lzs3a4m O00 
-76396509D 00 
-75846824D 00 
«430973900 GD 
.74146667D 00 
-73028457D 00 


-f1 791907) 00 
. 70313092D 00 
-68741216D 00 
-67049975D 00 
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Table 2 
(Continued) 
0.41236000D 00 0.10000000D 01 0O.10195757D 
0.41245000D 00 0.10000000D 01 0.10190083D 
0.41254000D 00 0.10000000D 01 0.10184162D 
0.41263000D 00 0.10000000D 01 0O.10178028D 
0.41272000D 00 0.10000000D 01 0O.10171713D 
0.41281000D 00 0.10000000D 01 0O.10165246D 
***%** BAND FROM GRID POINT T2060 B *xX*** 
0.41290000D 00 0.10000000D 01 0.10158656D 
0.41301000D 00 0.10000000D 01 0O.10150473D 
0.41312000D 00 0.10000000D 01 0.10142192D 
0.41323000D. 00... 0..10000000D. 01 -08.10133853D 
0.41334000D 00 0.10000000D 01 0.10125497D 
0.414345000D: 00° a -hO0Q00000D: 04 .« 0:-7041 7159D 
0.41356000D 00 0.10000000D 01 0O.10108873D 
0.41367000D 00 0.10000000D 01 0O.10100667D 
0.41378000D 00 0.10000000D 01 0.10092570D 
0.41389000D 00 0.10000000D 01 0.10084608D 
**%*** BAND FROM GRID POINT 8 TO Q xx 
0.41400000D 00 0.10000000D 01 0.10076803D 
0.41417000D 00 0.10000000D 01 0O.10065098D 
0.41434000D 00 0.10000000D 01 0O.10053884D 
0.41451000D 00 0.10000000D 01 0O.10043218D 
0.41468000D 00 0.10000000D 01 0O.10033147D 
0.41485000D 00 0.10000000D 01 0O.10023710D 
0.41502000D 00 0.10000000D 01 0.10014936D 
0.41519000D 00 0.10000000D 01 0O.10006849D 
0.41536000D 00 0.10000000D 01 0.99994657D 
0.41553000D 00 0.10000000D 01 0.99927970D 
***%*%* BAND FROM GRID POINT 9 TO 10 ***** 
0.41570000D 00 0.10000000D 01 0O.99868488D 
0.41598000D 00 0.10000000D 01 0.99786248D 
0.41626000D 00 0.10000000D 01 0.99723406D 
0.41654000D 00 0.10000000D 01 0.99679534D 
0.41682000D 00 0.10000000D 01 0.99654003D 
0.41710000D 00 0.10000000D 01 0.99646012D 
0.41738000D 00 0.10000000D 01 0O.99654624D 
0.41766000D 00 0.10000000D 01 0.99678792D 
0.41794000D 00 0.10000000D 01 0.99717386D 
0.41822000D 00 0.10000000D 01 0.99769208D 
**%**%* BAND FROM GRID POINT 10-30 11 *#*#K* 
0.41850000D 00 0.10000000D 01 0O.99833011D 
0.41895000D 00 0.10000000D 01 0.99957423D 
0.41940000D 00 0.10000000D 01 0O.10010407D 
0.41985000D 00 0.10000000D 01. 0.10026754D 
0.42030000D 00 0.10000000D 01 0O.10044248D 
0.42075000D 00 0.10000000D 01 0O.10062374D 
0.42120000D 00 0.10000000D 01 0.10080639D 
0.42165000D 00 0.10000000D 01 0O.10098581D 
0.42210000D 00 0.10000000D 01 O0O.10115775D 
0.42255000D 00 0.10000000D 01 0O.10131836D 
*%***%* BAND FROM GRID POINT 1120 12 ***** 
0.42300000D 00 0.10000000D 01 0.10146425D 
0.42345000D 00 0.10000000D 01 0O.10159251D 
0.42390000D 00 0.10000000D 01 0O.10170070D 
0.42435000D 00 0.10000000D 01 0.10178694D 
0.42480000D 00 0.10000000D 01 0.10184982D 
0.42525000D 00 0.10000000D 01 0O.10188847D 
0.42570000D 00 0.10000000D 01 0.10190250D 
0.42615000D 00 0.10000000D 01 0.10189199D 
0.42660000D 00 0.10000000D 01 0.10185744D 
0.42705000D 00 0.10000000D 01 0O.10179979D 
***%*%* BAND FROM GRID POINT 12-TO 13 4% 
0..42750000D- 00. .0.:70000000D 01...0.10172030D 
0.42805000D 00 0.10000000D 01 0O.10159582D 
0.42860000D 00 0.10000000D 01 0.10144452D 
0.42915000D 00 0.10000000D 01 0O.10127028D 
0..42970000D. 00...0..10000000D- 01... 0.1410107730D 
0.43025000D 00 0.10000000D 01 0.10087000D 
0.43080000D 00 0.10000000D 01 0O.10065288D 
0.43135000D 00 0.10000000D 01 0.10043042D 
0.43190000D 00 0.10000000D 01 0O.10020702D 
0.43245000D 00 0.10000000D 01 0.99986912D 
**%*%*%* BAND FROM GRID POINT 13-70 14 **** 


6. 


4-7 


9) 7° E'S CS 


(30 YE eam Mi ac Bl Se Es, oe IY 


YY Yi ea co Ry YR aD, a ca Pl ae: J | 


Cs Ym YO cobs WB) ca JN ck cate 1 > BR a PA a YK ca 


ee) eee 


ore iC Go Wee 1S) te 


eC) OE Cap > 


.65252410D 00 
-63360920D 00 
664387271D 00 
.59342618D 00 
sof a0.F521D 00 
.55081958D 00 


-52885347D 00 
-50157765D 00 
-47397244D 00 
-44617820D 00 
-41832464D 00 
~39053136D 00 
. 36290838D 00 
£2825 5056D 00 
.30856820D 00 
-28202742D 00 


-25601064D 00 
“21699227D 00 
.17961268D 00 
.14406032D 00 
-11049151D 00 
. 79033543D-01 
-49787442D-01 
.22830594D-01 
wieeneag 7 3D-02 
.24010040D-01 


. 43836976D-01 
.71250407D-01 
921981 69D-01 
~1:0682197D 00 
winte33237D 00 
wt799615D 00 
1131 2560D 00 
1 706939D. 00 
.94204776D-01 
~16930922D-01 


Soho 31 23D-01 
yA4I1924 99D- 01 
.34690503D-01 
.891:786/9D-01 
.14749376D 00 
.20791344D 00 
.26879637D 00 
eeeo0352D 00 
.38591673D 00 
-43945445D 00 


-48808469D 00 
-93083512D 00 
-56690020D 00 
-59564508D 00 
-61660646D 00 
-62949036D 00 
-63416685D 00 
-63066217D 00 
.61914826D 00 
.59993034D 00 


.57343289D 00 
.53193950D 00 
.48150652D 00 
.42342644D 00 
.35910105D 00 
.29000047D 00 
.21762531D 00 
.14347290D 00 
.69007842D-01 
.43628569D-02 
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** 


** 


** 


** 


* * 


CC Gi Ca Ca ae Ea Ca) Ca es GO Ore £) O'S CO C.o9 OG OO 2 Oo Ga OO ea OO Oe ka CoO OS Ca ama as oa Se OS So Ce Oe 2 Ol So © c ee © ec 
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Table 2 
(Continued) 
.43300000D 00 0.10000000D 01 0.99774069D 
.43356000D 00 0.10000000D 01 0.99568654D 
-43412000D 00 0.10000000D 01 0.99378179D 
-43468000D 00 0.10000000D 01 0.99205839D 
-43524000D 00 0.10000000D 01 0O.99054424D 
.43580000D 00 0.10000000D 01 0.98926318D 
.43636000D 00 0.10000000D 01 0.98823476D 
.43692000D 00 0.10000000D 01 0.98747431D 
.43748000D 00 0.10000000D 01 0O.98699283D 
-43804000D 00 0.10000000D 01 0.98679699D 
** BAND FROM GRID POINT 14 TO 15 4H 
.43860000D 00 0.10000000D 01 0O.98688913D 
.43917000D 00 0.10000000D 01 0O.98727659D 
-43974000D 00 0.10000000D 01 0O.98795361D 
.44031000D 00 0.10000000D 01 0O.98890878D 
.44088000D 00 0.10000000D 01 0.99012605D 
.44145000D 00 0.10000000D 01 0O.99158467D 
-44202000D 00 0.10000000D 01 0.99325934D 
-44259000D 00 0.10000000D 01 0O.99512014D 
.44316000D 00 0.10000000D 01 0.99713278D 
.44373000D 00 0.10000000D 01 0.99925879D 
** BAND FROM GRID POINT 15-“TO 16 *##** 
.44430000D 00 0.10000000D 01 0O.10014557D 
.44487000D 00 0.10000000D 01 O0O.10036776D 
.44544000D 00 0.10000000D 01 0O.10058758D 
.44601000D 00 0.10000000D 01 0O.10079990D 
.44658000D 00 0.10000000D 01 0.10099947D 
-44715000D 00 0.10000000D 01 O.10118103D 
.44772000D 00 0.10000000D 01 0.10133937D 
.44829000D 00 0.10000000D 01 0.10146952D 
.44886000D 00 0.10000000D 01 0O.10156687D 
-44943000D 00 0.10000000D 01 0O.10162735D 
** BAND FROM GRID POINT 16TO 17 #xKR* 
-45000000D 00 0.10000000D 01 0O.10164754D 
~.45045000D 00 0.10000000D 01 0O.10163331D 
-45090000D 00 0.10000000D 01 0O.10159152D 
~.45135000D 00 0.10000000D 01 0.10152176D 
.45180000D 00 0.10000000D 01 0.10142406D 
~45225000D 00 0.10000000D 01 0.10129894D 
.45270000D 00 0.10000000D 01 0.10114742D 
.45315000D 00 0.10000000D 01 0O.10097101D 
.45360000D 00 0.10000000D 01 0O.10077177D 
-45405000D 00 0.10000000D 01 0.10055223D 
** BAND FROM GRID POINT 1 eT 18 **#* 
-45450000D 00 0.10000000D 01 0O.10031541D 
-45495000D 00 0.10000000D 01 0O.10006481D 
.45540000D 00 0.10000000D 01 0.99804357D 
-45585000D 00 0.10000000D 01 0.99538348D 
.45630000D 00 0.10000000D 01 0.99271450D 
-45675000D 00 0.10000000D 01 0.99008619D 
~45720000D 00 0.10000000D 01 £0.98755054D 
-45765000D 00 0.10000000D 01 0.98516140D 
~.45810000D 00 0.10000000D 01 0.98297383D 
~-45855000D 00 0.10000000D 01 0.98104336D 
** BAND FROM GRID POINT 148° TO 19 *XHH* 
.45900000D 00 0.10000000D 01 0.97942525D 
~.45929160D 00 0.10000000D 01 0.97856902D 
.45958320D 00 0.10000000D 01 0.97788085D 
.45987480D 00 0.10000000D 01 0.97737438D 
.46016640D 00 0.10000000D 01 0.97706251D 
.46045800D 00 0.10000000D 01 0.97695714D 
.46074960D 00 0.10000000D 01 0O.97706901D 
.46104120D 00 0.10000000D 01 0.97740730D 
.46133280D 00 0.10000000D 01 0.97797941D 
.46162440D 00 0.10000000D 01 0.97879041D 
** BAND FROM GRID POINT 19 TO 20 ***** 
.46191600D 00 0.10000000D 01 0.97984262D 
.46208440D 00 0.10000000D 01 0O.98055988D 
.46225280D 00 0.10000000D 01 0O.98135653D 
.46242120D 00 0.10000000D 01 0.98223124D 
.46258960D 00 0.10000000D 01 0O.98318216D 
.46275800D 00 0.10000000D 01 0.98420682D 
.46292640D 00 0.10000000D 01 0O.98530202D 


ONG Cor Ca Coo" ee Se Co Ce eo Sa Sy Ca ae or GS Cr ey ar Ce Ga GD Sy) ean Can Ce CD Ca Ca Cae cS COGS Ca aaa ae Oo ¢ oS aa co Ge a oe 


Ca Conc Ca "Ca" 


.75310269D-01 
.14378228D 00 
.20727361D 00 
.26472054D 00 
.31519196D 00 
.35789432D 00 
.39217477D 00 
.41752302D 00 
.43357239D 00 
.44010034D 00 


-43702874D 00 
.42411345D 00 
-40154633D 00 
. 36970726D 00 
~seetd-1B1D 00 
.28051073D 00 
-22468875D 00 
-16266201D 00 
SFIS Pat +30 =O 1 
. 24707332D-01 


-48523386D- 01 
<t22586821TD 00 
.19585872D 00 
.26663173D 00 
.33315810D 00 
.39367734D 00 
-44645757D 00 
-48984066D 00 
~oe229130D 00 
.54244880D 00 


.54917973D 00 
-54443719D 00 
-53050826D 00 
.50725415D 00 
-47468723D 00 
-43298007D 00 
.38247160D 00 
.32367061D 00 
s2of2ce2ziD 00 
.18407528D 00 


.10513703D 00 
-21604847D-01 
.65214538D-01 
-15388379D 00 
-24284985D 00 
.33046045D 00 
-41498217D 00 
-49462008D 00 
sIOTIS9 11D: OG 
-63188776D 00 


-68582494D 00 
.71436620D 00 
.73730509D 00 
.75418718D 00 
.76458304D 00 
.76809520D 00 
-76436645D 00 
.75308967D 00 
-73401947D 00 
.70698625D 00 


-O7T9ITZSOD 00 
-64800384D 00 
-62144912D 00 
~99229199D 00 
-56059447D 00 
-52643932D 00 
-48993239D 00 
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Table 2 
(Continued) 
0.46309480D 00 0.10000000D 01 0.98646385D 00 
0.46326320D 00 0.10000000D 01 0.98768750D 00 
0.46343160D 00 0.10000000D 01 0.98896724D 00 
**%*%*%* BAND FROM GRID POINT 20 TO 21 **#**% 
0.46360000D 00 0.10000000D 01 0.99029633D 00 
0.46372330D 00 0.10000000D 01 0.99129625D 00 
0.46384660D 00 0.10000000D 01 0.99231493D 00 
0.46396990D 00 0.10000000D 01 0.99334844D 00 
0.46409320D 00 0.10000000D 01 0.99439249pD 00 
0.46421650D 00 0.10000000D 01 0.99544236D 00 
0.46433980D 00 0.10000000D 01 0.99649286D 00 
0.46446310D 00 0.10000000D 01 0.99753830D 00 
0.46458640D 00 0.10000000D 01 0.99857251D 00 
0.46470970D 00 0.10000000D 01 0.99958876D 00 
**%*%*%* BAND FROM GRID POINT Zt 22 **#KH 
0.46483300D 00 0.10000000D 01 0.10005797D 01 
0.46492460D 00 0.10000000D 01 0.10012949pD 01 
0.46501620D 00 0.10000000D 01 0.10019884D 01 
0.46510780D 00 0.10000000D 01 0.10026564D 01 
0.46519940D 00 0.10000000D 01 0.10032952D 01 
0.46529100D 00 0.10000000D 01 0.10039007D 01 
0.46538260D 00 0.10000000D 01 0.10044686D 01 
0.46547420D 00 0.10000000D 01 0.10049945pD 01 
0.46556580D 00 0.10000000D 01 0.10054736D 01 
0.46565740D 00 0.10000000D 01 0.10059010D 01 
**%**%* BAND FROM GRID POINT Z22-TO 23 ***** 
0.46574900D 00 0.10000000D 01 0.10062716D 01 
0.46584070D 00 0.10000000D 01 0.10065804D 01 
0.46593240D 00 0.10000000D 01 0.10068212D 01 
0.46602410D 00 0.10000000D 01 0.10069882D 01 
0.46611580D 00 0.10000000D 01 0.10070755D 01 
0.46620750D 00 0.10000000D 01 0.10070767D 01 
0.46629920D 00 0.10000000D 01 0.10069855D 01 
0.46639090D 00 0.10000000D 01 0.10067950D 01 
0.46648260D 00 0.10000000D 01 0.10064986D 01 
0.46657430D 00 0.10000000D 01 0.10060891D 01 
0.46666600D 00 0.10000000D 01 0.10055594D 01 
****%* BAND FROM GRID POINT 24 TO 25 4H 
0.49444400D 00 0. 0.53664005D-02 
0.49599960D 00 0. 0.23290146D-02 
0,49755520D 00 0. 0.57134124D-04 
0.49911080D 00 0. 0.19262916D-02 
0.50066640D 00 0. 0.33826998D-02 
0. 502222000 00 —0. 0.45079746D-02 
0.50377760D 06... 0. 0.53664970D-02 
0.50533320D 600 6. 0.60093980D-02 
0.50688880D 00 0. 0.64775363D-02 
0.50844440D 00 0. 0.68037441D-02 
***%*%* BAND FROM GRID POINT Feo We ig 26 **#*** 
0. 5100800CGR 60° —0. 0.70145375D-02 
0.51044800D 00 oO. 0.70569896D-02 
0.51089600D 00 0. 0.70921265D-02 
0.51134400D 00 oO. 0.71203702D-02 
3.511 792000 00-0. 0.71421190D-02 
0.51224000D 00 oO. 0.71577488D-02 
0.51268800D 00 0. 0.71676144D-02 
8.5343713600D 00 = 0. 0.71720504D-02 
0.51358400D 00 6, 0.71713735D-02 
GO. S7403200D 900° 6. 0.71658821D-02 
***** BAND FROM GRID POINT 26 TO 27 **¥#** 
0.51448000D 00 0. 0.71558586D-02 
0: STS032000 00-6; 0.71376699D-02 
0.51558400D 00 0O. 0.71134734D-02 
0. 516736000 00 <6; 0.70837068D-02 
0.51668800D 00 OO. 0.70487784D-02 
2.53 7260000 O80 — 4. 0.70090696D-02 
0. 547792000 00 .-0,. 0.69649370D-02 
0.51834400D 00 0O. 0.69167132D-02 
0.51889600D 00 0. 0 .68647096D-02 
0.51944800D 00 OO. 0.68092172D-02 
***%*%* BAND FROM GRID POINT 2: TO 28 *¥H** 
0.52000000D 00 OQ. 0.67505077D-02 
0.56800000D 00 oO. 0.22088895D-03 
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-45120503D 00 
-41041687D 00 
-36775870D 00 


-32345566D 00 
-29012479D 00 
-25616929D 00 
-22171894D 00 
-18691706D 00 
, TSA9g1 35D DO 
-11690478D 00 
-82056549D- 01 
-47582919D-01 
-13708201D-01 


.19324343D-01 
.43164152D-01 
.66279129D-01 
.88547527D-01 
.10984092D 00 
.13002403D 00 
.14895458D 00 
.16648317D 00 
.18245317D 00 
.19670060D 00 


-20905412D 00 
-21934500D 00 
-22/37191D 00 
-23294069D 00 
+23584995D 00 
-23589117D 00 
-23284891D 00 
~.226501T12D 00 
-21661944D 00 
-20296961D 00 
»18531197D 00 


-53664005D 00 
-23290146D 00 
-57134124D-02 
- 19262916D 00 
-33826998D 00 
-45079746D 00 
-53664970D 00 
-60093980D 00 
-64775363D 00 
-68037441D 00 


-70145375D 00 
-70569896D 00 
.70921265D 00 
»71203702D 00 
-71421190D 00 
-71577488D 00 
-71676144D 00 
-71720504D 00 
ett FPS Fao OO 
-71658821D 00 


-71558586D 00 
.71376699D 00 
-71134734D 00 
- 70837068D 00 
-70487784D 00 
-70090696D 00 
-69649370D 00 
691677 32D--00 
-68647096D 00 
-68092172D 00 


-67505077D 00 
-22088895D-01 
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Or Cari): Cncotaarce 


THE 


THE 


.61600000D 
-66400000D 
. 71200000D 
.76000000D 
. 80800000D 
-85600000D 
.90400000D 
.95200000D 
. 10000000D 


VALUE OF A FROM THE FINE GRID IS 


FINAL VALUE OF THE ERROR ON THE FINE GRID IS 


Ore: OV: © OxO''O'© 


Table 2 
(Continued) 
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.31906682D-02 
-48591221D-02 
ps At Be Se Bo Bak ED 3 
631 BIA SD Ne 
.66508561D-02 
-68657264D-02 
.69986081D-02 
107 12714D-02 
. 70944158D-02 


C1 | Oe" "aa" 


.31906682D 
-48591221D 
ed f A Aa OD 
63 1.2430 5D 
.66508561D 
-68657264D 
.69986081D 
.70712714D 
. 709441 58D 


0.7501062336D-02 


0.7680951936D 00 
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Cepstral Analysis 


A. V. Oppenheim 


Introduction 


This section consists of two programs related to the computation of the cepstrum and the generation 
of the minimum phase equivalent of a nonminimum phase signal. The cepstrum method is a nonlinear 
signal analysis technique which has found application to a variety of problem areas including echo 
detection [1], speech analysis [2], geophysical data processing [3] and a variety of others. Computation 
of the cepstrum is also an intermediate step in homomorphic filtering for deconvolution [4]. 


The cepstrum is defined as the inverse Fourier transform of the logarithm of the Fourier transform 
of the input sequence and, in general, this requires the evaluation of the phase as a continuous function 
of frequency. This in turn requires computation of an "unwrapped" phase, i.e., a phase curve for which 
the discontinuities associated with computation of the phase modulo 27 are removed. The cepstrum 
obtained in this manner is referred to as the complex cepstrum since it requires the use of the complex 
logarithm. 


An alternative form of the cepstrum is based on the use of the logarithm of the magnitude rather 
than the complex logarithm of the Fourier transform. This form of the cepstrum is often referred to as 
the real cepstrum. For a minimum phase signal the complex cepstrum is easily obtained from this form 
of the cepstrum by simple windowing in the cepstral domain and the need for phase unwrapping is 
thereby eliminated. Computation of the cepstrum using only the magnitude of the Fourier transform 
followed by appropriate windowing in the cepstral domain and the inverse cepstral transformation also 
provides a means of transforming a signal to its minimum-phase equivalent. A discussion of the 
theoretical details can be found in Oppenheim and Schafer [4]. 


This section contains two programs related to cepstral analysis. The first program is directed at the 
computation of the complex cepstrum and contains as a major component the program for phase 
unwrapping. It should be noted that the phase unwrapping program included in Section 7.1 has many 
other applications in addition to its importance in cepstral analysis. For example, phase unwrapping 
arises in the context of tracking propagation times in the ocean using sinusoidal sources. The second 
program computes the real cepstrum and implements the appropriate windowing and inverse transfor- 
mation to obtain the minimum-phase reconstruction of the signal. In applying these programs and cep- 
stral analysis in general, it is often important to avoid data which has been oversampled since this will 
introduce a frequency band with small amplitudes and low signal-to-noise ratio which will lead to 
unreliable phase unwrapping and will tend to dominate the cepstrum because of the logarithmic 
transformation. Thus the use of modulation and decimation/interpolation filtering may be required 
prior to cepstral analysis. This and a variety of other computational issues and concerns are discussed 
in detail in Tribolet [3]. 
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1. Purpose 


The program is designed to compute the complex cepstrum X(n) of a real sequence x(n). The 
complex cepstrum is defined as the inverse Fourier transform of the complex logarithm of the Fourier 
transform of the input sequence. 


2. Method 


The complex cepstrum X[n] of a real sequence x[n] is evaluated by means of the DFT computa- 
tional realization as discussed by Oppenheim and Schafer [1] 


X[n] = IDFT(ogIDFT (x[n]))) (1) 


where the DFT length is sufficient to avoid cepstral aliasing. The complex logarithmic operation 
involved in this computation makes it necessary to evaluate the phase of x[n] as a continuous function 
in frequency. This evaluation, called phase unwrapping, is accomplished by means of the adaptive 
numerical integration algorithm proposed by Tribolet [2]. This technique uses both the principal value 
of the phase and the phase derivative to compute the unwrapped phase at each frequency. The evalua- 
tion of the phase derivative requires the knowledge of the DFTs of both x(n) and nx(n). 


The basic idea of the phase unwrapping algorithm is the following. Let 0, be an arbitrary frequency 
value, ARG[X ie? ')] be the principle value of the phase, arg[X i" ')] be the unwrapped phase, and 
arg'[X(e’"")] the phase derivative at 0. The set of permissible phase values at 1, is given by 


{ARGIX(e')] + 2a, / integer} (2) 
The phase unwrapping problem amounts to determining the correct integer value /.(,) such that 
arg(X(e’"")] = ARGIX(e’™)] + 2a/.(Q)) . (3) 


This is done through the use of numerical integration of the phase derivative. We adopt here the tra- 
pezoidal integration rule. Assuming the unwrapped phase to be known at a frequency Ny) < 0), we 
define a phase estimate at 0, arg[X(e° 8 | Qo] by 


o,- 


arg [X(e’")| Qo] = arg[X(e’™)] + © [arg'[X(e7™)] + arg'[X(e’™)]] . (4) 


Cepstrum Programs Chapter 7 


Clearly, this estimate improves as the step interval AQ = 2, — {o becomes smaller. We define the 
phase estimate at 0, to be consistent if it lies within a predefined distance of one of the permissible 
phase values at 0,, that is, if there exists an /,({,) such that 


latg(X(e/™)| O51 — ARGIX(e’*")] + 2m/,(Q,)|<THLCON<z . (5) 


The basic idea of this algorithm is thus to adapt the step size A(M until a consistent phase estimate is 
found. The resultant /.(,) in Eq. (5) is used in Eq. (3) to form the unwrapped phase at 0. This 


unwrapped phase is then used to form a7g|X (e 92) |0,], ©, > Q4, and so on. 


For this algorithm to be practical, one must take full advantage of the FFT algorithm and reduce the 
number of extra discrete Fourier transform (DFT) computations to a reasonably small number. Let us 
denote by 


{w,=(2a/N)k, k=0,1,....N—1} (6) 


the set of uniformly spaced frequencies with interval 277/N where N = 2” (or, in general, any highly 
composite number). The phase derivative and the principal value of the phase at these frequencies 
may then be computed using FFT’s to evaluate the DFT’s of x(n) and nx(n). At each w,, a phase 
estimate is initially formed by one-step trapezoidal integration, starting at w,_}. If the resultant esti- 
mate is not consistent, the adaptive integration scheme is applied within the interval [w,-1,@,]. The 
step size adaptation was carefully designed to minimize the number of extra DFT’s required. The 
search for consistency is done by consecutively splitting the step interval in half. As the required phase 
derivatives and principal values are computed, they are stored in a stack fashion. As soon as a con- 
sistent estimate is found, the corresponding data are moved out of the stack to a register that holds the 
most recent consistent estimate of the phase at some frequency within [w,_;,,]. New estimates are 
always formed by integrating from the most recent estimate to the frequency corresponding to the top 
of the stack. 


3. Program Description 


The desired sequence x(n) is generated within the main program CCMAIN through subroutine 
COEFF. A diagram of this mainline program is given in Fig. 1. The FFT length and threshold levels 
THLCON (of Eq. (5)) and THLINC (see Section 5 on computational accuracy for a description of this 
threshold) are initialized. The computation of the complex cepstrum is accomplished by calling subrou- 
tine CCEPS and proceeds as follows. First, the FFTs of both x[n] and nx[n] are computed. Then, the 
log spectral magnitude, the principal value of the phase, and the phase derivative of x[n] on the FFT 
frequency grid are evaluated. Next, adaptive phase unwrapping is successfully performed between these 
equispaced frequencies, through the use of function PHAUNW. A block diagram of this program is 
given in Fig. 2. After the successful completion of the unwrapping, the linear phase component is 
removed and an IFFT taken, to yield the complex cepstrum x[n]. A block diagram of CCEPS is given 
in Fig. 3. Within CCMAIN the first and last 32 values of the complex cepstrum are then printed, along 
with its corresponding sign and linear phase component. 


The code for these routines and additional secondary routines is given in the Appendix where 
dimension requirements are specified. 
4. Test Problem 


In order to facilitate testing, as well as help the user getting acquainted with the potentialities and 
limitations of this program, the subroutine COEFF is provided for generation of test signals of finite 
length, with prescribed z-transforms. 


Section 7.1 
MAIN PROGRAM: CCMAIN 


Specify input 
x [n],n=0,1...NX-I 


Specify: 
e FFT length, NFFT 


e Phase consistency threshold, THLCON 
e Phose increment threshold, THLINC 


Compute 
complex cepstrum: 
CCEFS 


Was 


CCEPS 
successful: 
ISSUC=.TRUE. ? 


NO YES 


Fig. 1 Block diagram of main program: CCMAIN. 


CEP 


The specifications included in CCMAIN will generate a sequence x[n] whose z-transform X(z) has 


6 zeros at z,,Z}" ,2),Z) ,23,Z3, where: 
aie (0.9) 7/4) 


iy(— + ——)) 
za= (hie 4 oo 

t+. )] 
z3= (0.9)e 4 8192 


The output of CCMAIN is given in Table 1. The results match those evaluated analytically, using 


the equation 


ze tezht tt zee 


bP iron errter are a 


n<0 


(7) 
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FUNCTION PHAUNW 


Stack initialization 
Update previous 
estimate 


Evoluote phase 


Compute phase 


derivative ond increment 
principle value of across 
phose at intermediate spectrol 


frequency interval 


Is 
increment 
greoter than 
THLINC? 


Form phase 
estimate 


Define intermediote 
frequency by halving 
spectral interval 


phase estimate: 
ISCONS= 
.TRUE.? 


ISCONS=.FALSE, 


RETURN 


Fig. 2 Block diagram of Tribolet’s adaptive phase unwrapping algorithm. 
5. Computational Accuracy 


It is important to realize that when the input data x(n) has a number of very sharp zeros near the 
unit circle (although not on it), the computational noise generated, for example, in the FFTs, may lead 
to erroneous values for the principal value of the phase and for the phase derivative near such zeros. 


Subroutine PHAUNW is designed to detect such cases and interrupt the unwrapping process, typing 
the message "Phase Estimation Failed". This detection is controlled by the values THLINC and 
THLCON. 


THLCON is defined by Eq. (5) of Section 2. and, for most seismic and speech signals of our 
interest, is set at 0.5. THLINC is used to detect the possibility of an erroneous 271.(0,) of Eq. (3) in 
a region of a sharp zero; that is, a poor integration step within Eq. (4) may lead to a consistent estimate 
but in error by a 27 multiple. Setting THLINC to 1.5 will avoid any such phenomenon, by requiring 
further step adaption, when the unwrapped phase increment is greater than 1.5. However, in certain 
circumstances the stack memory may not be sufficient for the given thresholds. 


For example, if the magnitudes of z,,z, and z3 in the example above are 0.9999, 1.0001 and 0.9999, 
respectively, then the unwrapping fails. 


The authors tested a version of this program on an ECLIPSE minicomputer, using precisely the 
code published in this book, except in that its compilation (including the FFTs) was done using the 
Eclipse-Fortran 5 Double Precision Compiler Option. This has the effect of forcing all real variables 
and constants to become double precision. Furthermore, all single precision Fortran library functions 


7.1-4 


Section 7.1 CEP 


SUBROUTINE CCEPS 


Compute FFTs of 
x(n} and nx {nj 
Define signal polority, ISNX 


Compute on FFT frequency grid: 
eSpectral magnitude 
ePhase derivative 

e Principle value of phase 


Compute oat each FFT frequency: 
eLog- magnitude 
eUnwrapped phose: 
FUNCTION PHAUNW 


Step FFT 
frequency 


Wos — 
unwrapping 
consistent: 
ISCONS= 

TRUE.? 


ISSUC=.FALSE. 


ISSUC = .TRUE. 


Compute signal lag, ISFX 


Remove linear phase component 


Compute complex cepstrum, x [n]: 
IFFT 


RETURN 


Fig. 3 Block diagram of program for computation of complex cepstrum. 


having double precision counterparts are replaced by them. 


Such double precision version of this code was capable of correctly unwrapping the data of the 
example above. An important issue in the usage of this program is therefore the selection of threshold 
levels THLINC and THLCON. 


Small values of these thresholds are associated with high probability of error detection, and thus 
with greater confidence in the program’s output. Unfortunately, the use of small threshold values is 
also associated with longer computation times. 


The user is thus strongly encouraged to run this program under a variety of test signals, so as to 
determine the accuracy requirements and maximum threshold levels that might be appropriate for his 
particular application. 
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Table 1 Output of main program: CCMAIN. 


7.1-6 


INPUT SIGNAL-LENGTH (IN SAMPLES) = a 
X ( 1)= 1.0000 x({ 2)= -4.0996 X( abe, 6.4057 xf 4)=-10.1765 
X ( Spm. 7. 7801 Ki 6)= -3.5142 xX( Vis: 0.7939 xf 8)= 0. 
SIGN= 1 
LINEAR PHASE= -2 
COMPLEX CEPSTRUM 
CX( 993)= -0.0030 CX( 994)= -0.0024 CX( 995)= -0.0000 CxX( 996)= 0.0030 
CX( 997)= 0.0050 Cx( 998)= 0.0040 cx{ 999)= 0.0001 CX(1000)= -0.0052 
Cx(1001)= -0.0085 CxX(1002)= -0.0069 Cx(1003)= -0.0001 Cx(1004)= 0.0090 
Cx(1005)= 0.0149 Cx(1006)= 0.0123 CxX(1007)= 0.0001 CxX(1008)= -0.0164 
CX(1009)= -0.0272 CxX(1010)= -0.0227 CX(1011)= -0.0002 CX(1012)= 0.0314 
CX(1013)= 0.0531 Cx(1014)= » 0.0453 Cx(1015)= 0.0003 Cx(1016)= -0.0664 
CX(1017)= -0.1166 CX(1018)= -0.1040 CxX(1019)= -0.0004 CX(1020)= 0.1753 
CX(1021)= 0.3415 CxX(1022)= 0.3546 CX(1023)= 0.0006 CX(1024)= -1.2852 
CX +)= 0. T906 2)= -2.5446 CxX( a)= 0700712 Cx{ 4)= 0.6881 
S}= 0.6501 6)= 0.3334 cx( 7)= -0.0008 CxX( S)=--—0..1938 
9)= -0.2152 10)= =-0. 429S°C2U 17)= 0.0005 CX 12)= 0.0810 
13)= 0.0941 14)= 0.0550 cx( 15)= -0.0004 Cx( 16)= -0.0390 
17)= -0.0463 18)= -0.0276 CX( 19)= 0.0002 CX( 20)= 0.0203 
21S UUeZtS 22)= 0.0146 CX( 23)= -0.0002 CxX( 24)= -0.0110 
25)2 -0,0733 26)= -0.0080 CX( 27)= 0.0001 CxX( 28)= 0.0062 
29)= 0.0075 30)= 0.0045 Cx( 31)= -0.0001 CX( 32)= -0.0035 
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and 


J. M. Tribolet 


Instituto Superior Tecnico 
Lisbon, Portugal 


1. Purpose 


The program is designed to compute the real cepstrum and the minimum-phase reconstruction of an 


arbitrary phase, real sequence x(n). The real cepstrum is defined as the inverse Fourier transform of 


the log magnitude of the Fourier transform of the input sequence. 


2. Method 


The real cepstrum x(n) of a real sequence %(n) is evaluated by means of the DFT computational 
realization as discussed by Oppenheim and Schafer [1]. 


&(n) = IDFT (log|DFT(x(n)]) (1) 
where the DFT length is sufficient to avoid cepstral aliasing. 


For the minimum-phase reconstruction of a sequence x[n], the real cepstrum X[n] is windowed by 
win): 


0 n.< 9 
w[In] =41 n= (2) 
2. wa > D 


With x(n) having a rational ztransform, this windowing operation is equivalent to mapping 
maximum-phase poles and zeros to their conjugate symmetric counterparts within the unit circle of the 
z-plane. Finally, the minimum-phase reconstructed sequence y(n] is computed by inverse complex 
cepstral mapping $[n] = X[n]wI[n]. This mapping is evaluated as follows: 


y[n] = IDFT(exp[DFTQI[n]))) (3) 
The inverse DFT length must likewise be sufficient to avoid aliasing in the time domain. 


The inverse mapping may also be used after windowing the real cepstrum or complex cepstrum to 
accomplish separation of two convolutionally combined sequences whose cepstra are additively com- 
bined. For example, a "low-time" rectangular cepstral window is often applied in speech processing to 
retrieve the cepstrum of the vocal tract impulse response, and a "high-time" window is used in seismic 
processing to recover the earth’s reflector series (Oppenheim and Schafer [1]). 


3. Program Description 


The three steps involved in the computation of the forward mapping are respectively, an FFT, 
determination of the log magnitude spectrum, and an IFFT. This task is performed by subroutine 
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SUBROUTINE: RCEPS SUBROUTINE: ICCEPS 
Transform cepstral 
sequence, cx(n): 
FFT 


Exponentiate 
the log- 
spectrum 


Tronsform input 
sequence, x(n): 
FFT 


Compute log- 
magnitude; set 
phose to zero 


Inverse 
Transform: 
IFFT 


Inverse 
Transform: 


RETURN RETURN 


Fig. 1 Block diagram of subroutine RCEPS. Fig. 2 Block diagram of subroutine ICCEPS. 
RCEPS, whose flow diagram is depicted in Fig. 1. 


The three steps involved in the inverse mapping are respectively, an FFT, complex exponentiation, 
and an IFFT. This task is performed by subroutine ICCEPS, whose flow diagram is depicted in Fig. 2. 


The main program RCPICP first computes the real cepstrum of a specified sequence and FFT length 
by calling RCEPS. The first and last 32 values of this sequence are printed. Next, the minimum-phase 
reconstruction is determined, after appropriate windowing, by ICCEPS, and the first and last 32 values 
are again printed. 


The FFT lengths of the forward and inverse computations within RCPICP are equal, but in general 
these lengths can be tailored to the particular aliasing problems. A flow diagram of RCPICP is given in 
Fig. 3. 


The code for these routines and additional secondary routines are given in the Appendix along with 
array dimension requirements. 


4. Test Problem 
The input x[n] is a sequence whose z-transform X(z) contains one zero outside the unit circle: 
X(z) = (1.+(1./0.99) z7!) (4) 


The output of RCPICP is given in Table 1. These results are consistent with the analytical evaluation 
for both the real cepstrum and the minimum-phase reconstruction 


X[0] = log(1./0.99) 
X[n] = —(—0.495)"_ |[n| > 0 (5) 
1.0101 n=0 
Xmintn] = 41.0000 n=1 
0.0000 elsewhere 
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MAIN PROGRAM: RCPICP 


Specify input 
x [n],n= 0,1... NX- | 


Specify forward, 
inverse FFT 
lengths 
NFFTF ond NFFTI 


Compute real 
cepstrum: 
RCEPS 


Output real 
cepstrum 


Cepstral filtering: 
Find cepstrum 
of 
minimum-phose 
reconstruction 


Compute inverse 
cepstrum: 
ICCEPS 


Output 
minimum-phose 
reconstruction 


Fig. 3 Block diagram of main program RCPICP. 


References 


1. A. V. Oppenheim and R. W. Schafer, Digital Signal Processing, Chapter 10, Prentice-Hall Inc., 
Englewood Cliffs, New Jersey, 1975. 


7.2-3 


Cepstrum Programs Chapter 7 


Table 1 


KH 


REAL CESTRUM 


RCPCIP- TEST PROGRAM OUTPUT *** 


Cx( 993)= -0:01% “CK 994) = 5<OwO1 2 Cet 995) =: -0.012.. cxf 996)= 0.073 
Cx 9987)= -0.0T3 998)= 0.014. ¢cx( 999)= -0.015 Cx(1000)= 0.016 
CX¥(1001)= -0.016  CxX(71002) =. 0 O77 Ce (1003) = =0.018 ¢€x(1004)= 0.079 
CxX(1005)= -0.020 1006)= 0.022 Cx(1007)= -0.023 Cx(1008)= 0.025 
CX(1009)= -0.027 1010)= 0.029 cx(1011)= -0.031 Cx(1012)= 0.034 
©X(1013)= -0.037 4074)= 0.604%. ¢CX(1015)= -—0.045. -6X(1016)= ~ 0.051 
Cx(1T017)= -0.058 cCx(1018)= .0.067 Cx(1019)= -0.078 Cx(1020)= 0.095 
Cx(10215= -0.120 CE(7022)= Gotee.. *(1023)= =0.245- Cx 1024) = 0.495 
cx }2. “00 2y=--- 0495 CX ( 3)= -0.245 cx( 4)= 0.162 
cx 5 ie ih 6)= 0.095. ck 7)= -0.078 cx( 8)= 0.067 
Cx 9)= -0.058 105= UUs) CXU 1 11)= -0.045.-CHt  “T2y= 0.04% 
Cx. 1372 -tsa7 14)= WO OX 2 15)= -0.031° CR(  16)= 0.029 
Cx( -17)= -Ga27 1R)= UFOS" "Cx(: 19)= -0.023 Ext Z20)= 0.022 
CX¥(- 2ti2:=§.020 Daya CEC 23)= -0.018 Cx -24)= 0.017 
CX(- 2532: -0.916 26)= 0.016 CX( 27)= -0.015 Cx{ 28)= 0.014 
CK 29)2--0.013 o0i= 0.0 eC. 31)=-0.012 Cxl 32)= 0.032 
MINIMUM-PHASE RECONSTRUCTION 
TCx{993)= 0.000 994)= -0.000 ICX( 995)= 0.000 ICx( 996)= 0.000 
ICK( 997)= 0.000 ICcx( 998)= -0.000 ICx( 999)= -0.000 ICx(1000)= -0.000 
Icx(1001)= -0.000 I¢cx(1002)= -0.000 1¢xX(1003)= -0.000 ICxK(1004)= -0.000 
Icx(1005)= -0.000 Icx(1006)= 0.000 1¢x(1007)= -0.000 ICcx(1008)= 0.000 
TCX (1009)= 6.000 ICK(1010)= -0700021CK(1011)= -0.000 Icx(1012)=. 0.000 
TICK (1073)=— 05000 101472" 207-000" TCX (1015) = =0.000- 1CX(1076)= 0.000 
TCK( 1077) =.- 6.009 1018)= -0.000 ICcx(1019)= -0.000 ICx(1020)= 0.000 
ICX(1021)= -0.000 1022)=""0.000 TCx(1023)= 0.000 ICx(1024)= -0.000 
ICX ( a =e oy 2 tin0G. oe ( 3}=. 0.000 -1Cx( 4y=-- 0.0800 
TCR Sie 8. 0e8 6).=....8...000..ICxX( y= > 02000 -2Ck( 8)= =-0.000 
Text 9)= -0.000 40)= -0.000 IcxK(  11)= -0.000 ICX( 12)= -0.000 
ICRC 13 = = ee 14)= -0.000 Icx( 15)= -0.000 Icx( 16)= 0.000 
TCX { 17) = -0..000 18)= 0.000 ICx( 19)= -0.000 ICx( 20)= 0.000 
ICxyt 21} = ee 22)= 0.000 Icx( 23)= -0.000 Icx( 24)= 0.000 
IGE( .-25)= -02008 26)= 0.000 Icx( 27)= -0.000 ICX( 28)= 0.000 
ICx( :29)= =0,000 S0;= 0.000 1cet 31)8 =0. 000 1Cx{ 32)= 0.000 
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CHAPTER 3 


Interpolation and Decimation 


R. E. Crochiere 


Introduction 


The processes of interpolation and decimation are fundamental operations in.digital signal process- 
ing. They are used whenever it is necessary to change from one sampling rate to another in a digital 
system. For example, in speech processing, estimates of speech parameters are often computed at a 
low sampling rate for low bit-rate storage or transmission. This operation can often be conveniently 
defined in terms of concepts of decimation. For reconstructing a synthetic speech signal or a decoded 
replica of the original speech from the low bit-rate representation, the speech parameters are normally 
required at much higher sampling rates. In such cases, the sampling rate must be increased by a digital 
interpolation process. 


The techniques of interpolation and decimation are also useful in defining highly efficient ways for 
implementing narrow band FIR filters and modulators. By careful design these techniques can be used 
for bandpass and highpass filters as well as the traditional lowpass filter. Computational efficiencies 
comparable to those of recursive elliptic filters are possible while still retaining the desirable properties 
of linear phase response and limit-cycle free operation. Yet another application of decimation and 
interpolation concepts can be found in digital phase shifting and in implementations of fractional sample 
delays. 


Two types of computational issues generally arise in the design and implementation of interpolation 
and decimation systems. The first issue involves the design of appropriate filters around which the 
decimation or interpolation is based. The second issue involves the actual implementation of the deci- 
mation or interpolation processing. Three programs are provided in this section, in addition to the pro- 
grams in other sections, which are useful in accomplishing these tasks. 


The design of interpolation or decimation filters involves the use of lowpass (or bandpass) digital 
filters. Such filters can be designed in a variety of ways, e.g. window designs, equiripple designs, etc. 
For many applications this design can be achieved by the use of the filter design programs in Chapter 5. 
However, for interpolation designs with mean square minimization of interpolation error and preserva- 
tion of original signal values, specialized procedures exist. 


Program 8.1 is one such example in which this mean square error criterion is used. The program 
applies to the design of interpolation filters in which the sampling rate is increased by an integer factor 
of R. The resulting filter design allows the original input samples to pass through the interpolator 
unchanged and it interpolates R—1 sample values between each pair of original samples in such a way 
that the mean square error between these samples and their theoretically ideal values is minimized. 
The design is achieved through the formulation of an L XL matrix inversion where 2L is the number of 
original sample values used to interpolate each new sample. The resulting interpolation filter is a 
2RL +1 tap, linear phase, FIR, digital filter. The size of L is limited to be 10 or less and the program 
requires the use of a matrix inversion program which is provided. 


The implementation of interpolators or decimators involves the implementation of digital filters in 
which the input and output sampling rates are different. Because of this difference in Sampling rates a 
straight-forward implementation of FIR digital filter structures is not practical. Instead, special 
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considerations must be taken in interfacing the different sampling rates and in efficiently implementing 
the digital filter. The next two programs in this section provide alternative approaches to this problem 
based on whether the ratio between the two sampling rates is of the order of unity (say for example 5/8 
or 16/15) and can be expressed as a ratio of integers, or whether the ratio is a very large integer value 
which can be factored into smaller integer values (for a cascade implementation). 


Program 8.2 covers the first category in which the ratio of output to input sampling rates can be 
expressed as L/M where L and M are arbitrary integers. In the case where M = | an integer interpola- 
tion by a factor of L is performed and when L = | an integer decimation by a factor of M is performed. 
The program requires a coefficient array of a symmetric FIR decimating or interpolating filter which 
must be supplied by the user (any of the above designs can be used). After an initialization call, data 
can be processed in a consecutive block-by-block manner by repeated calls to the program. Data 1s 
received and returned in blocks whose sizes are related by the ratio L/M to accomodate the difference 
in input and output sampling rates. 


Program 8.3 covers the second category of decimation and interpolation implementations in which 
the ratio of output to input sampling rates is a large integer (in the range 15 and up) which can be fac- 
tored into smaller integer values. Generally it is more efficient in this case to perform the interpolation 
(or decimation) by a series of cascaded stages of interpolators (or decimators). In this way the filter 
requirements and computation at each stage can often be substantially reduced over that of a single 
stage implementation. For example a 1:100 interpolator might be more efficiently implemented by a 
three stage cascade of interpolators with conversion rates of 1:2, 1:5, and 1:10 than by a single stage 
conversion. For such a case, speed improvements of up to 50 can be obtained. The filter designs for 
each of the stages must be supplied by the user. As in Program 8.2, after an initialization call, data is 
processed in a consecutive block by block manner with unequal input and output block sizes to accomo- 
date the different input and output sampling rates. The program can implement 1, 2 or 3 stage decima- 
tor or interpolator designs. It also has the capability of implementing a cascade of a (1, 2, or 3 stage) 
decimator and a (1, 2, or 3 stage) interpolator for the efficient implementation of narrowband FIR 
filters. 


In addition to the programs and methods presented here, a number of other techniques and design 
methods for decimation and interpolation have been proposed in the literature. Bellanger et aes gat A & 
for example, have proposed design techniques based on single stage or cascaded stages of 2:1 decima- 
tors or interpolators with "half band" digital filters. In this case advantage can be taken of the fact that 
half of the coefficients in the digital filter are exactly zero. Goodman and Carey [3] have proposed a 
number of specialized, low order, filter designs for cascaded stages of decimators or interpolators. 
Parks and Kolba [4] have recently proposed a new minimax filter design for interpolation to comple- 
ment the least square designs of Program 8.1. 
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A Computer Program for Digital Interpolator Design 


G. Oetken 


Institut fiir Nachrichtentechnik 
Universitat Erlangen-Nirnberg 
Cauerstrasse 7 
8520 Erlangen, Germany 


T. W. Parks 


Department of Electrical Eng. 
Rice University 
Houston, TX 77001 


H. W. Schiissler 


Institut fiir Nachrichtentechnik 
Universitat Erlangen-Niirnberg 
Cauerstrasse 7 
8520 Erlangen, Germany 


1. Purpose 
This program [1] designs a length 27L + 1 FIR interpolating filter with unit pulse response h(n), 
n = —TrL,...,rL, which, given every r-th sample of a signal x(n), interpolates the remaining samples. 
2. Method 
The unit pulse response / (n) is designed so that 
IGe-|_ | |) * — xll? 


is minimized for bandlimited signals x (7). The symbol * represents convolution and - represents mul- 
tiplication. The symbol | | E represents sampling, 


7 1 n=0 mod r 
L 1 J,(n) = 0 otherwise 
The signal x(n) is assumed to be bandlimited with 
X(m) =0 for |w| >aa/r. 


The squared norm of the error is 


lell?= SY e(n)?. 
The method divides h(n) into r subsequences h .(/) = h(rl+r), 7 = 0,1,...,r—1 and solves the follow- 
ing set of linear equations for each 7 


=: h(ri+7)b(r(I—m)) = b(rm+r) m =-L,...,L—-1 
[=—L 


aT/r 
where o(k) = f |X (w)|*e~**dw is the autocorrelation of x. In terms of matrices these equations 
—atr/r 


become 
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Rather than invert the 2Lx2L matrix ®, the algorithm, using symmetries of ®, inverts 2 LxL 
matrices. These inverse matrices are then used to calculate only the first half of the pulse response 
h(—rL),...,h (0) since h(n) is an even function. 


3. Program Description 
3.1 Usage 

The subroutine DODIF requires subroutine SMINVD and a user defined function DPH1. 
3.2 Subroutines and Functions 


SUBROUTINE DODIF (L,R,ALPHA,DIF) is called from a main program and designs the optimal digi- 
tal interpolating filter. 


SUBROUTINE SMINVD (A,N,EPS,IER) inverts positive definite symmetric matrix using double preci- 
sion. 


DOUBLE PRECISION FUNCTION DPH1 (X,ALPHA) calculates the autocorrelation of the input data. 
3.3 Description of Parameters 


R — Wanted increase in sampling rate, i.e., the filter calculates R—1 samples between each pair of 
input samples. Range for R is any integer >1. 


L - Integer, L, which determines the degree of the filter. The degree is 2-L:R. Range for L is 
1 to 10, may be increased at the risk of numerical instability (change LMAX and array 
bounds of AP,AM,D for L > 10). 


ALPHA — Cutoff frequency (radians) of the filter input data divided by 7/R. Range for ALPHA is 
0 < ALPHA < 1.00 (double precision). 


DIF — Double precision array in which the first half (L-R+1 values) of the unit sample response is 
returned. (The optimal filter has linear phase.) 


A - Double precision upper triangular part of given positive definite symmetric N x N matrix 
stored columnwise. On return A contains the upper triangular part of the resulting inverse 
matrix in double precision. 


N - The number of rows (columns) in the given matrix. 
EPS — Relative tolerance for test on loss of significance. 
IER — Resulting error parameter IER = —1 indicates error in inverting matrix possibly due to loss 


of significance. 
xX — Point for which the autocorrelation is desired. 
3.4 User Requirements 


The user must supply the main program and the autocorrelation function DPH1. In the main pro- 
gram the dimension of DIF must be fixed at a value greater than or equal to L-R+1. 


Big values of L and small values of ALPHA increase the possibility of problems with matrix inver- 
sion (depending upon the wordlength of the computer). A very good possibility to control the quality 
of the matrix inversion is to check the magnitude of the unit sample response DIF(K) at 
K = M-R +1, M = 0to L-1, which ideally have to be zero. 

4. Test Problem 
A test program was run to design an interpolating filter for data with a flat spectrum, i.e., 
\X¥(w)|2=1 for lol < az/r. 
The parameters were R = 4, L = 4, and ALPHA = 0.500. 
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Table 1 


OPTIMAL INTERPOLATING FILTER DATA 


L= 4 
THERE ARE 
H(-16)= 0 
H(-15)= -0 
H(-14)= -0 
H(-13)= -0 
H(-12)= -0 
H(-11)= 0 
H(-10)= 0 
HY -8)= 0 
H{ -8)=. =0 
Bi <7 je. --0 
Bt -6)2..-0 
Bt =3e <0 
H(-4)= <0 
H( -3)= 0 
H( -2)= 0 
H{ -=1)" 0 
0)= 0 


-4559319E-02 
-6777514E-02 
-5177760E-02 
.8881784E-15 
.2578440E-01 
-3945777E-01 
-3118661E-01 
-8881784E-15 
-8770110E-01 
-1426581E 00 
-1220465E 00 
~-3552714E-14 
-2910058E 00 
-6098364E 00 
-8713054E 00 


-1000000E 01 


ALPHA= 0.50 


17 VALUES IN HALF OF THE FILTER 
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A General Program to Perform Sampling Rate 
Conversion of Data by Rational Ratios 


R. E. Crochiere 


Bell Laboratories 
Murray Hill, NJ 07974 


1. Purpose 


This program can be used to convert the sampling rate of data by factors of L/M where L and M are 
arbitrary positive integers. Input data is provided through an input buffer in blocks of length NpM and 
output data is received through an output buffer in blocks of NpZ where Np is any positive integer. 


2. Method 


The program is based on the one stage interpolator/decimator method of R. E. Crochiere and L. R. 
Rabiner [1,2]. Conceptually the method is depicted in Fig. 1. 


etal INCREASE DECREASE y(n) 
SAMPLING SAMPLING 
fr RATE BY L RATE BY M L f, 


Fig. 1. Sampling rate converter 


The input sampling rate f, is increased by a factor L by inserting L—1 zero valued samples between 
each pair of input samples. This signal is then filtered with an FIR lowpass filter whose stopband cutoff 
frequency is f,/2 or Lf,/(2M), whichever is smaller. The output signal of the filter is then reduced by 
a factor of M by keeping only one out of every M samples. By the elimination of multiplications by 
zero and elimination of computing unnecessary output samples it has been shown in [1] that the output 
y(n) can be computed by the relation 


T (1) 


y(n) = 7 h(kL + (nM) ® L)x 


k=0 


eek 


where ()@ZL implies the quantity in parentheses modulo L and [] corresponds to the integer less than 
or equal to the number in brackets. The sequence /(n), n =0,1,...,N—1 represents the coefficients of 
the FIR filter and NV is the number of taps in the filter such that 


N SQL (2) 
and Q is an arbitrary positive integer. 


Figure 2 shows the flow diagram of the program which implements relation (1). Input data x(n) is 
supplied through BUFM and output data y() is received through BUFL. QBUF stores the necessary 
internal state variables and COFS stores the coefficients 4(n) in a prescribed "scrambled" order such 
that they can be sequentially accessed (see [1]). ICTR is a control memory which is generated by the 
initialization program and is used to control the indexing of data and coefficients in the program. 


The group delay of the sampling rate converter at the Output sampling rate is 


eo hag 
no ks (M—-1) 


(samples) (3) 
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INPUT OUTPUT 
BUFFER BUFFER 


BUFM BUFL 


Q—ee- Qs fe 0 
t 4 h(n) (SCRAMBLED) 


INDEX CONTROL OF 
QBUF UPDATE AND 
COFS SELECTION 


Fig. 2. Flow diagram of program 


where it is assumed that the FIR filter is a linear phase, i.e., symmetric, filter. If a zero delay conver- 
Sion is required, the filter length N can be selected to make 7, an integer and the first 7, output sam- 
ples can be discarded. 


3. Program Description 
3.1 Usage 


The program consists of a main program (supplied by the user) and two subroutines, one for initializa- 
tion (SRINIT) and one for processing (SRCONV). On initialization the user supplies values for the 
parameters M, L, NQ, COEF, N, NC and NI. For processing the user supplies values for BUFM, 
BUFL, and ND. Once the program is initialized SRCONV can be used as often as necessary for pro- 
cessing contiguous blocks of data. 


3.2 Description of Parameters 


M Decimation ratio (see Fig. 1), ©. 

L Interpolation ratio (see Fig. 1), L. 

QBUF Internal buffer for storage of state variables of filter. 

NQ Size of QBUF, equal to or greater than 2*Q, where Q is the next greatest integer 
of N/L. 

COEF Buffer containing FIR symmetric filter coefficients. Because of symmetry the 


buffer should contain only half of the filter coefficients beginning at the middle 
impulse response coefficient. 


N No. of taps in FIR filter, WN. 
COFS Buffer for storing scrambled coefficients. 
NC Size of buffer COFS, equal to or greater than L*Q where Q is the next greatest 


integer of N/L. 
ICTR Buffer for storing control data generated by SRINIT and used by SRCONV. 
NI Size of ICTR, greater than or equal to 2*L. 


IERR Error code for debugging 
= 0 Noerrors found in initialization 
= 1 QBUF (NQ) too small 
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= 2 COFS (NC) too small 
= 3 ICTR (NI) too small 
BUFM Input data buffer for processing, size Ny* MM. 
BUFL Output data buffer for processing, size Np*L. 
ND Any positive integer, Np. 
3.3 Dimension Requirements 


The DIMENSION statement in the main program should be modified according to the requirements of 
each particular problem. The required minimum dimensions of the arrays are 


QBUF must be of dimension equal to or greater than 2* (the next greatest integer of 
N/L). 


COEF _ must be of dimension [(N+1)/2] where [x] is the integer part of x. 


COFS must be of dimension Q*L where Q is equal to or greater than the next largest 
integer of N/L. 


ICTR must be of dimension 2* L. 
BUFM must be of dimension Np*M. 
BUFL must be of dimension Np* L. 
3.4 Summary of User Requirements 
(1) Specify M, L, NQ, N, NC, NI 
(2) Fill in coefficient buffer COEF 
(3) Adjust dimensions for QBUF, COFS, ICTR, COEF, BUFM, BUFL 
(4) CALL SRINIT once for initialization 
(5) Supply BUFM with input data 
(6) CALL SRCONV and obtain output samples in BUFL 
(7) Repeat steps (5) and (6) as often as desired 


4. Test Problem 


Compute the unit sample response of a sampling rate converter which converts data from a sampling 
rate of 10 kHz to 16 kHz. 


The ratio of conversion is 16/10 or preferably 8/5. Therefore, L = 8 and M =5. The input sampling 
rate f,, is 10 kHz and the output sampling rate Lf,/M is 16 kHz. As seen from Fig. 1 the intermediate 
sampling rate Lf, is 80 kHz which corresponds to the effective sampling rate of the lowpass filter. 
Since 10 kHz is the lowest sampling rate involved, the lowpass filter should have a stopband cutoff of 
one half of this rate or 5 kHz. The filter was designed using the McClellan, et al. program [3]. 


The test program reads in the filter coefficients from the standard input unit and outputs them to the 
standard output unit. It initializes the conversion routine by calling SRINIT and then generates a unit 
sample in the BUFM array by zeroing it out and setting the first sample value to one. It converts the 
sampling rate of this array by the factor 8/5 by calling SRCONV. BUFL then contains the interpolated 
impulse response of the sampling rate converter at the 16 kHz sampling rate. Figure 3a shows a plot of 
this response. 
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Fig. 3. Impulse response, frequency response, and group delay of test example. 


Figure 3b shows the log magnitude of the frequency response obtained by taking the Fourier transform 
of Fig. 3a. The filter characteristics are clearly evident. Figure 3c shows the group delay of the sam- 
pling rate converter. Note that the delay is 11 samples as predicted by Eq. (3) and is exactly flat with 
frequency because of the linear phase (symmetric) FIR filter. 


Table 1 shows the output of the test program for the case L = 8, M =5, and N = 119; the coefficients 
of the 119 tap FIR filter input to test program are echoed back in the output of the test example. Also 
the unit sample response of Fig. 3a is given in Table 5 
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Table 1 

COEFFICIENTS 
098344 096741 .092027 .084489 074579 
.062882 050073 .036869 .023975 .012040 
001611 .006898 .013228 017283 019127 
.018966 047423 014002 .010055 005738 
.001477 002359 005488 .007720 .008970 
009248 008652 007342 005525 .003429 
001279 000722 002410 .003668 004435 
004700 004502 003916 .003043 001998 
000896 000157 001076 .001799 .002292 
002545 002576 002416 002112 .001715 
601377 .000844 000452 .000129 .000112 
000268 000345 000357 000319 .000533 

IERR= 0 

UNIT SAMPLE RESPONSE 

-.002144 010217 .020362 007169 037602 
.010230 .073986 .011818 151726 012884 
.503055 786755 .503055 012884 .151726 
-~.011818 .073986 .010230 .037602 007169 

.020362 010217 002144 000000 - 


Chapter 8 


Interpolation 


UALNIOd SSHYddV DNIAOW JO dNLgas O dOLs 
2 (ASNOdSHU AIdWVS LINN HLZ) LYWHOd 7666 
ANNILNOD O€ (€I *‘=uYuYsHI H9) LVWHOT S666 
ANNILNOO 02 ( HL) LYWHYOd 9666 
TE ae eae 8 (9°OLHS) LYWHOI L666 
°O = (1)S400 (40ON°LO°WW) AI (SLNZIOIddAHOO HEL) LVWHOAU 8666 
Ta*(WW)dHOD = (I)SHOD (dJON'AIT'WNW) JI (O°LLdL) LVWHOU 6666 
(L-dON-N) - XW = WW (JON’°AD° XW) AI (NN°L=MN°(M)TANGA) (LE66°Ld1) ALIUM 
XW - JON = WW (JON’LI'XW) JAI ve = NN 
ole oe. a an A ll (7666‘'Ld1) ALIUM 
OI‘ L=OW 02 Od (ULOI *‘Sdod ‘ando ‘aN ‘14nd *‘WANA)ANOOUS TIVO 
T° L=IW O€ OG € = GN 
L=tI1 * 
2 VLVG SSH00uUd OD 
SLNALOLIddaOO AIAWVYNOS OD 3 
2 "L = (L)Wand 
GANNILNOD OL ANNILNOD OL 
0 = (I) ando "0 = (1I)WANG 
ONS L=I OL Od GL°L=I OL OG 
D a 
andd LAO OUaZ DO FTIdNVS LINN ALVYANAD O 
D> 2 
Te (9666‘Ld1T) ALIUM 
Z/(L+N) = JON UUAI (S666SLd1) ALIUM 
C= Wael (dN LION) al (weet “18 “wLOT ‘ON *Sa0D ‘Nn *aG0D “ON * 480°] “NI GINIES TIVD 
L = WYRI ((O1*7Z)°LI°ON) AI O€ = ON 
TxOI = dN G = W 
b+ OL 8 OE (U(i*Or)SN°N) ar G =.4 
I/N = OL 9L = IN 
2 OZL = ON 
OL ALNdWOD OD 6LL = N 
> 4 
et ANILNOW NOISUXANOD AZIIVILINI DO 
0 = WueI 2 
2 (9666°Ld1) ALIYM 
TIVWS OOL (IN) ULOI € = o (NN‘L=4M°(M)dHO9) (LE66°LdT) ALIUM 
TIVWS OOL (ON) SHOD Z = oD (8666‘Ld1T) ALIYM 
TIVWS OOL (ON) JNdd | = a (NN‘L=4N‘(M)dH09) (6666°QNI) Avau 
NOILVZIIVILINI NI GNNOd SUYONUR ON O = , 09 = NN 
ONIOONGAG YOd AGOD YOUUT = AAAI O (Z)HOWNLI = Ld 
lT*Z NVHL YALVAND YO IWNOA YOLOTA ULOI AO AZIS = IN 2 (L)HOVWWLI = ANI 
ANOOUS AG GHSN GNV LINIUS AG AALVYANAD AVUUV TIOULNOD = UALOI O 5 
(I/N dO USOALNI LSALVEYD LXAN AHL) *71 ) (Ld1) LINO LNdLAO GUYVGNVLS SHL NO NE3LLIYM SI LAdLNO TIvV :LNdLNo 5 
NVHL UYALVEYD YO OL IVNOA *YOLOAA SdMOD JO AZIS = ON 2 i 
LINIYS AG GALVYANAD YOLOAA LNALOIAdg9OD AAIAWVYOS = SAHOO OD (GNI) LINO LOdNI GUVGNVLS AHL WOUA AVEAN AYV SLNAIOIdMAGOO :LNdNI O 
UALIIA ONILWIOdUALNI YId AHL NI SdvVL dO ‘ON = N 9 ‘8 
UALIIA ONILVIOGUSLNI YIa WOA SLNAYLOIAAIOO JO AVUUNV = AOD O TINd WOUA VLVG LAdLNO SNINVL ANV WANd HONOMHL VLVd LAdNI OD 
(I/N dO UADALNI LSALVAYD 2 ONIAIddNS ANOOUS STIVO NAHL GNV SZIIVILINI OL LINIYS STIVO WVYSOUNd AHL O 
LXON AHL)*Z OL IWNOT YO ULVAEUS ‘Ando AO AZIS = ON se) W/I dO OILLVUY V AG IVNDIS V dO ALVY ONITIMNVS AHL SLYSANOD WVuDONd SIHL O 
UAIING ATIAVIUVA ALVLS = ANAD O 2 
OILVY NOILVIOdYALNI = 7 3 pa ae 
OLLVY NOILVWIORd = W 2 0= or 
2 0 = OI 
(L)MLOI *‘(L)SHOO *‘(L)a909 6(L)4Nd@O NOISNAWIG (SL) Wand 
TI ‘OL ‘OI /WODYS/ NOWWOD ‘(pz) Tana *‘(9L)UYLOI ‘(0€)4Ndd £(07ZL)SHOD £(09)dHOD NOISNAWIG 
(YuaI * TI ‘Or ‘OI /WOOUS/ NOWWOD 
‘IN ‘YLOI ‘ON ‘SHOO ‘N ‘dH90D ‘ON SANGO ST ‘W)LINIYS ANILNOUNS 5 
DQ pe eee essa 5 
sioaterhontbeatantctantiaeatentaarionteariaanbartentasteet asarietantentendiatenttanteattentatant anteater ieatatententatentaa tea actatarterteatertest eaten kestesten aaa techies D2 UALIId ONILVIOGUALNI YIld YOI SLNAIOIAGAHOO AO AVAeUV = AOD >LNdNI O 
W/1l dO OILVY AHL AG IVNDIS WV dO DO PL6OLO AMSUAC MAN ‘TITIIH AVUMAW ‘SHIYOLVUYORVI TIA 3 
ALVY ONIIMWVS AHL SLYZANOD HOIHM ANOOUS YOA NOILVZIIVILINI 9D » @UYRIHDOND A U >YOHLAV OD 
be. LINIYS :ANILNOWENS D OILVU IVNOILVY V Ad ny 
pe te tt ie a a i ae ae 1s Sad aa nA a ae a NOISUSANOOD ALVUY ONIIMWVS WHOdUNAd OL WVHYDONd LSAL :WVYDOUd NIVW DO 
QD ne en rs oD 
aNa fe) 


xIpueddy 


8.2-6 


SRCONV 


Section 8.2 


aNd 
NYN LAY 
ANNILNOO 
ANNILNOO 
Loe a Se 
WOS = (aT) TANd 
ANNILNOO 
(GOOI)SHOO* (dOI)ANdO + WAS = WAS 
Of + ON = aol 
oi +) O28. = gO 
OI‘ L=O0N O€ Od 
'O = WOS 


THNA NI GHYOLS AGNV VLVG LNdLNO dO AIdNWVS | ALNdWOO OD 


OL OL OD 

Le eae Sat 

OL =. Of (0 OR OL) ae 
by ODEN ee 

L + GN = GN 


(dW)WAINd = (LOL) ANAO 
DI + OL = tor 
(QW)WHnNnd = (Of) Ando 


OZ OL OD (0°OR'aL) AI 


£* kt = 3% 

Cb48)BLOL | = 21 

(M)ULOI = ar 
4° tsf: OF: 0d 
L=%H% 


STIaWVS LNdLNO I aLNaWOd 
Wand NO XHCGNI = aW 


GN‘ lt=I 0S Od 
yf Sa eae | 
lie 
L = dW 


ANOOUS AG GASN ANV LINIYS AG GCALVYANAD AVUNV 'IOULNOD = ALOI 
LINIYS AG G4LVYANAD YOLOGA LNALIOIaAdaYOO AAIAWVHOS = SAOD 
YdddHNd ATAVIAYVA ALVLS = ANdo 

UHDaALNI AAILISOd ANV = AN 
TI*CN @ZIS dO WaddGNnd VLIVd LNdLNO = Tand 
W*CN AZIS dO WdadNnd VLVd LNdNI = Wand 


(L)YLOI *(L)SHOD *‘(L)andd *‘(L)Tdn@d *‘(L)Wand NOISNAWIG 
TI ‘OL ‘OI /WOOYUS/ NOWWOD 
(ULOI *‘SdOO ‘dando ‘aN ‘14nd ‘W4N@)ANOOUS ANILONoOwns 


“ANILNOW SIHL ONITIVO OL YOIUNd GATIVO Ad LSOW LINIUS 
“W/T OILVY AHL Ad IVNOIS V dO ALVA DONITMWVS AHL SLYYANOO 
ANOOUS -aNILNoOwdns 


aNg 
NUOLaY 

GONILNOD 0S 

L + OW = OW 

0 8 OE 49'S ST) at 
L +99 = 37 OF 

L+u=H 

L- = OW 

Oo = wpMy 

0S Ot*o) © (ayeLor 

OF 

YOLOAA SAHOO NI NOILVOOT ONILUVLS 


O€ [ +3 = 3 
0 = YONI 
MONI = (A)ALOT 


GNdO ALVddN OL SHIdWVS JO ON 
02 
2) OF OL OD ((L-W)°LI°OW) AI 
L + MONT = WONT (0°OR°O1) al 
e) WI‘ L=I 0S Od 
Lt = 2 
0 


oo Nn 
il 
Oo 
= 


€ = Gal ((I*e) LT IN) at 
W*T = WI 


ULOI AVYUV TIOULNOO ALVYANAS 
OL 
oO Ul © OF 


ANd ALVddN oO 


OOO 


Oo 000 000 


OO OO 00 © 


OUD 


(7 0-0 


8.2-7 


ap. 


dee 
_ 


1a 


8.3 


A Program for Multistage Decimation, 
Interpolation, and Narrow Band Filtering 


R. E. Crochiere and L. R. Rabiner 


Acoustics Research Dept. 
Bell Laboratories 
Murray Hill, NJ 07974 


1. Purpose 


This program decimates, interpolates, or narrow band filters an input signal using a multistage reali- 
zation with up to 3 stages for both decimation and interpolation. The output sequence is written 
directly over the input sequence. 


2. Method 


The program is based on the method of R. E. Crochiere and L. R. Rabiner [1,2,3]. The user can 
decimate and interpolate (by an integer ratio) or narrow band filter (by a combination of integer deci- 
mation and interpolation) a signal by performing these operations in a series of stages. Up to three 
stages can be used for decimation and interpolation. For each stage a decimation ratio is required as 
well as a set of filter coefficients (of a direct form linear phase FIR digital filter) for the filtering to be 
performed in that stage. A block of samples of the input sequence is supplied and a block of samples 
of the output sequence is generated by the program. Thus to process a large sequence of samples the 
input sequence is partitioned into blocks and the program is called once for each block of samples. 


A flow diagram for the implementation of a three stage decimator in cascade with a three stage 
interpolator is shown in Fig. l(a) and its corresponding control sequence is given in Fig. 1(b). 
Together the cascade results in the implementation of a narrow band lowpass filter. To realize only a 
decimator or interpolator, appropriate parts of this structure are partitioned off (in the program) from 
the main structure. 


The decimator has three data buffers S1D, S2D, and S3D for storage of internal data for its three 
stages. These storage buffers are of durations N’,, N’,, and N’; words respectively. These lengths are 
derived according to the relationship 


N',=Q,D, 2 N; fmt 2.3. (1) 


where Q; is the integer greater than or equal to N/D,, N; is the filter length for the i * stage and D, is 
its decimation ratio. Each data buffer is partitioned into Q, blocks of data of length D,;. Three addi- 
tional buffers hold the coefficients for the filters in each of the three stages. The interpolator has six 
data buffers associated with it, three (SII, S2I, and S31) of lengths Q,, Q>, and Q; respectively and 
three (T1I, T2I, and T3I) of lengths D,, D>, and D; respectively. In addition, each stage i has a buffer 
for the "scrambled" coefficients which are partitioned into D, blocks of Q; samples each. The operation 
of this structure is depicted by the control sequence in Fig. 1(b). The process begins by reading D, 
samples from the main I/O buffer into S1D. One output sample is then computed from stage one of 
the decimator and stored in S2D. This process is repeated D, times until D, samples have been com- 
puted and stored in S2D. One output is then computed for stage 2 of the decimator and stored in S3D. 
The above process is repeated D; times until D3 samples have been stored in S3D at which point one 
output sample is calculated from stage 3 of the decimator. This completes one cycle of the decimator; 
D = D,D,D; samples have been read from the main I/O buffer and one output sample has been com- 
puted. A similar computation cycle can now proceed for the interpolator. The output sample from the 
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en SZ) COEF. VECTOR 
COEF. VECTOR a L : : = —— 
{ ae Oy —e lame Uj { 
—=s—— Ny SAMPLES — 05“ | SaMPLES| SAMPLES | SAMPLES | SAMPLES 
Ni 
<_<—_—_———_N,’ SAMPLES ———> 
STAGE 10 > N,; S 
1 
Q; 
S{I Sane STAGE 
——Wy—N>-' SAMPLES —————_ 
D 0 D2 — =» 
mm 2 ee | ae 2 Om | dc S20 a 
T2I 
Qe2 
{ 


ho(n) a ho(n) (SCRAMBLED) 
COEF. VECTOR cS COEF. VECTOR 
“~S—No SAMPLES —= SQ) we 29 — pe | ame — O90 —ee |e — 92 — 
No a SAMPLES | SAMPLES | SAMPLES 
STAGE 2D = —=———_ N,' SAMPLES ———=> 


s2I |<— Qe 


Ee A 
SAMPLES gna et 


—Nz SAMPLES ———= 


—_—_—_—_-_—___— 
-«-D; = —<D5—> olee yee S$3D ee D3 = 
SAMPLES| SAMPLES | SAMPLES | SAMPLES SAMPLES 
hg (n) S) 2 hz (n) (SCRAMBLED) 
COEF. VECTOR a COEF. VECTOR 


—<s— Nz SAMPLES —= rf SO és ON ici Qs - 
3 gS SAMPLES SAMPLES 


STAGE 3D <= Nz’ SAMPLES ———> 
—— S oe 
$31 SAMPLES STAGE 31 
RO ee Se MN EET eee RE Sf RE ee OE 
DECIMATION INTERPOLATION 


Fig. 1(a) Flowchart for a 3 stage decimator followed by a 3 stage interpolator. 


decimator is stored into S3I. D,; output samples are computed from stage 3 of the interpolator and 
temporarily stored in T3I. One sample from T3I is then stored in S2I and D, output samples are com- 
puted from stage 2 and stored in T2I. SII is then updated by one sample from T2I and D, outputs are 
computed and stored in the main I/O buffer. This is repeated until all D, samples in T2I are removed. 
T2I is then refilled by updating S2I with one sample from T3I and computing D, more samples. Upon 
completion of the interpolator cycle one input was transferred into the interpolator and D output sam- 
ples are computed and stored in the main I/O data buffer. The process can then be repeated on the 
next block of D samples. 


Because of the delay associated with the linear phase filters the output (decimated, interpolated, or 
filtered) waveform will have a delay associated with it with respect to the input. This delay can be 
determined as follows: let 


NPS 


5 (samples) 


TNI = 
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START 


UPDATE SID 
BY D; SAMPLES 
FROM MAIN I/O BUFF 


COMPUTE { OUTPUT 
SAMPLE FROM STAGE 1D 
& STORE IN S2D 


DO D3 
TIMES 


DECIMATION 
Te ee COMPUTE 1 OUTPUT 
SAMPLE FROM STAGE 2D 
& STORE IN S3D 
COMPUTE 1 OUTPUT 
SAMPLE FROM STAGE 30 
& STORE IN S31 
REPEAT UNITIL 
MAIN I/O BUFF 
IS FILTERED 
COMPUTE D3 SAMPLES 
IN STAGE 3I 
& STORE IN T3I 
UPDATE S2I 
BY | SAMPLE 
FROM T3I ) 
TIMES 
COMPUTE D2 SAMPLES 
IN STAGE 2I 
INTERPOLATION & STORE IN T2I 
1 SAMPLE 


TO D SAMPLES 


UPDATE SII 
BY | SAMPLE 
FROM T2I 


COMPUTE D, SAMPLES 


IN STAGE 1I & STORE 
IN MAIN I/O BUFF 


DONE 


Fig. 1(b) The control sequence for Fig. 1 (a). 


Nj+1 
Tix = — (samples) 


and 


N3+ 1 
Ti = — (samples) 


be the delays (group delay) of the filters for stages 1, 2, and 3. If only two stages are used let T,3 = 0 
and if only one stage is used let Ty7= 7y3= 0. The group delay, 7 i, of the multistage decimator at 
the output (the low sampling rate) is then 


] 


= Do (Ty, + DT n2+ D\D.7y3— (D\D,D3-1)} (Samples) 


TL 


(For decimator output) 
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where D; = 1 for a two stage implementation and D, = D; = 1 for a one stage implementation. The 
group delay for the interpolator output (at the high sampling rate) is 


TH=TNi+DiTN2+ D\D2T yz (samples) 
(For interpolator output) 
and for the filter (at the high sampling rate) 
Ty = WUT yytDyit yt D D7 y3) — (D\D2D3-1) (samples) 
(For the filter output) 
For the decimator and interpolator the group delay at the high sampling rate 7 ,, is related to the group 


delay at the low sampling rate, 7,, by the expression T, = D,D,D37T,. The duration of the overall 
impulse response is approximately twice the group delay. 


3. Program Description 
3.1 Usage 


The program consists of a main program (supplied by the user) and two subroutines, one for initiali- 
zation (DIINIT), and one for performing the required computation(DIFILT). The user is required to 
specify the number of stages, the decimation (interpolation) ratio for each stage, and a set of filter 
coefficients (of a direct form FIR linear phase filter) for each stage. In addition, the user supplies an 
input-output buffer to hold both the input and output sequences, and a scratch buffer of sufficient size 
to store all the internal buffers required in the computation. An error return is used to indicate any 
difficulty in initializing the package. Otherwise no output (other than the output array) is provided. 


3.2 Subroutines Required 


SUBROUTINE DIINIT (KD, ID1D, ID2D, ID3D, NID, N2D, N3D, COEF1, COEF2, COEF3, 
ITYPED, BUFF, IDJD, SBUFF, ISBD, IERR), called from main program, initializes the decimation- 
interpolation package. SUBROUTINE DIFILT (COEF1, COEF2, COEF3, BUFF, SBUFF) is called 
from the main program, once for each buffer of input samples, performs the required computation 
(i.e., decimation, interpolation, or narrow band filtering) and returns the output sequence in SBUFF. 
Assume BUFF is of size LD where L is an integer and D = D,D,D3. For a D:1 decimator fill BUFF 
with LD input samples and receive as output the first L samples in BUFF. For an interpolator supply 
only L samples to the first L locations in BUFF and receive as output LD samples. For a filter supply 
LD samples for input and receive LD samples as output. 


3.3 Description of Parameters 
KD Number of stages of decimation (and/or interpolation) 
ID1D Decimation ratio for stage 1 (i.e., D;) 
ID2D Decimation ratio for stage 2. (i.e., D>) 


ID3D Decimation ratio for stage 3 (i.e., D3) 
(ID3D = 1if KD = 2 and ID2D = ID3D = 1 if KD = 1) 


NID Filter duration (in samples) of linear phase FIR digital filter for stage 1 
N2D Filter duration (in samples) of linear phase FIR digital filter for stage 2 
N3D Filter duration (in samples) of linear phase FIR digital filter for stage 3 


(N3D = Oif KD = 2 and N2D = N3D = Oif KD = 1) 


COEF1 Buffer containing filter coefficients for stage 1 
COEF2 Buffer containing filter coefficients for stage 2 


COEF3 Buffer containing filter coefficients for stage 3. 
If the filter duration is odd, the program automatically changes COEFn(1) to COEFn(1)/2 
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for n = 1,2,3. Also COEFn contains half of the filter ramen response beginning at the 
middle impulse response coefficient. 


ITYPED 


BUFF 
IDJD 
SBUFF 


ISBD 
IERR 
IERR 


= 1 for a decimator alone 
= 2 for an interpolator alone 
= 3 for a narrow band filter (decimator followed by interpolator) 


Storage buffer for input and output sequence (Required dimension given below) 
Dimension of BUFF. (See below.) 


Storage buffer for internal variables, and scrambled coefficients for interpolation 
stages). (Required dimension given below.) 


Dimension of SBUFF (see below). 
Error Return for initialization (useful for debugging purposes). 


= (Q Noerrors 

= 1 Means KD is out of range, i.e., KD > 3 or KD < 1 
= 2 Means dimension for BUFF is too small or not an _ integer multiple of 
(D *D *D3) 

= 3 Means dimension for SBUFF is too small. 


3.4 Dimension Requirements 


The DIMENSION statement in the main program should be modified according to the requirements 
of each particular problem. The required dimensions of the arrays are 


COEF1 
COEF2 
COEF3 
BUFF 


SBUFF 


ISBD 2 (N1P+N2P+N3P) + (ID2D+ID3D) + 2*( 


must be of dimension [(N1D+1)/2] where [x] is the integer part of x 
must be of dimension [(N2D +1)/2] 
must be of dimension [(N3D +1)/2] 


must be of dimension IDJD which is computed by the rule | 

IDJD = L*x(ID1D#ID2D*ID3D) 
where L is any nonzero integer, i.e., IDJD is an integer multiple of the total decima- 
tion (interpolation) ratio of the system. 


must be of dimension ISBD which is computed by the rule: 
(a) For decimator only (ITYPED = 1) 
ISBD > 2*(NIP+N2P+N3P) 


where 
NIP = (J + 1)#IDID 
N2P = (Ince ] + 1)4ID2D 
N3P = (IS >] + 1)#ID3D ; 


a 
and [x] corresponds to the integer part of x (i.e., NIP, N2P, N3P correspond to ~ 
NN’, 4 =4,2, 3 in Ba. Ud: 


(b) For interpolator only (ITYPED = 2) 


NIP is N2P 4_N3P 
ID1D ID2D = ID3D 


where NIP, N2P, and N3P are defined above in (a). 


) 


8.3-5 


Interpolation Chapter 8 


(c) For narrow band filter ITYPED = 3) 
ISBD = ISBD , + ISBD i 


or 
decimator pd es 
(a) b) 


3.5 Summary of User Requirements 
(1) Specify KD, ID1D, ID2D, ID3D 


(2) Choose filter lengths NID, N2D, N3D and fill in filter coefficient buffers COEF1, COEF2, 
COEF3 


(3) Adjust dimension for BUFF and SBUFF 

(4) CALL DIINIT once for initialization 

(5) Supply BUFF with input data 

(6) CALL DIFILT and obtain output samples back in BUFF 

(7) Repeat steps (5) and (6) as often as necessary for continuous processing on blocks of IDJD sam- 
ples. 

4. Test Problems 


1. Decimate the allpass signal 


0 n<0O 
x(n) = —a n= 0 
(i-g55" ee 


for a = 0.9, by a factor of 10 to 1 using a 2 stage decimator with decimation ratios of 5 to 1 (for stage 
1D - Fig. 1) and 2 to 1 (for stage 2D). The filter impulse response durations are 25 samples for stage 
1D, and 28 samples for stage 2D. 


The main program will print out the filter coefficients for each stage, and will print out both the 
input sequence, and the decimated output sequence. The reader should note that the input sequence is 
decimated using 3 calls to the decimation subroutine in which consecutive blocks of 20 samples are 
decimated to give consecutive blocks of 2 samples. 


The parameters for this example are: KD = 2, IDID = 5, ID2D = 2, ID3D = 1, NID = 25, 
N2D = 28, N3D = 0, ITYPED = 1, IDJD = 20, ISBD = 106. 


2. Interpolate the allpass signal of example 1 using a 2 stage interpolator with interpolation ratios of 
1 to 2 (for stage 2I+) and 1 to 5 (for stage II). 


The filter impulse responses are the same as used in example 1. The main program will print out both 
the input sequence, and the interpolated output sequence. The reader should again note that the input 
sequence is interpolated using 3 calls to the interpolation subroutine in which consecutive blocks of 2 
samples are interpolated to give consecutive blocks of 20 samples. 


The parameters for this example are: KD = 2, IDID = 5, ID2D = 2, ID3D = 1, NID = 25, 
N2D = 28, N3D = 0, ITYPED = 2, IDJD = 20, ISBD = 106. 


3. Filter coefficients for a one stage decimator (or interpolator) with a decimation ratio of 10 to 1 are 
printed out to test out the one-stage implementation. 


Filter coefficients for a three stage decimator with decimation ratios of 10 to 1, 5 to 1 and 2 to 1 (for 
stages 1D, 2D and 3D respectively) are printed out to test the three-stage implementation. 


The output for these three examples including coefficient values, are contained in Table 1. 


+ The reader will recall that interpolation stages are numbered in reverse order because of their duality with decimation stages. 
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OUTPUT OF TEST 


COEFFICIENTS FOR STAGE 1 
- 168728 - 158866 - 131836 
-.012024 -.014156 -.011033 
COEFFICIENTS FOR STAGE 2 
. 352610 e237 1 -022946 
-011124 -.007792 -.009267 
INPUT SEQUENCE 
eee I Se LAD: ee 
sotse 86.0662  70596-- .0527 
DECIMATED OUTPUT SEQUENCE 
-0000 -.0003 
INPUT SEQUENCE 
<O257  .0231--,0208 .01@7 
»0089 .0081 .0072.---.0065 
DECIMATED OUTPUT SEQUENCE 
804-1: = 0617 
INPUT SEQUENCE 
~V031 0028: , 0825 0023 
09077 ..0010... 0009 -.69¢6e2 
DECIMATED OUTPUT SEQUENCE 
—.<0004 .0079 
INPUT SEQUENCE 
=, 9000 ~.4900 


INTERPOLATED OUTPUT SEQUENCE 
~. 0000 .-.0000 =.0000 -.0000 


.0006 .0005 -.0003. -.0001 
INPUT SEQUENCE 
Pe | es 


INTERPOLATED OUTPUT SEQUENCE 
-,.0055 =~. 0057 —-.0055 —-.0048 


<OT26 -. OTe ete ete 
INPUT SEQUENCE 
<ASOS: . 1287 


INTERPOLATED OUTPUT SEQUENCE 
~.0063 -.0146 -.0234 -.0321 
=, 0366 —. 0247 -=.0Ge1 .0091 


COEFFICIENTS FOR A 1-STAGE DESIGN 
.072497 at 2 a .068543 .064731 
.040717 033539 .026336 -UTS326 
=. 0939232 .006889 ,009636 -.017463 
=P? 009672 007882 -.005918 
.001423 - 002713 .003690 -004343 
.004091 -O03513 .002830 -002094 
-, 000513 .000939 -001247 -.001440 
-.001284 .001101 -000898 -.000692 
-.000069 .000007 .000049 -000506 
COEFFICIENTS FOR A 3-STAGE DESIGN 
STAGE 1 
089335 .086183 - 980139 .071694 
028149 .018410 <OTOT92 sU03719 =. 
-.006050 -005730 -.004931 -.003918 -. 
+=. 000715 -000364 -.000156 -.000052 


PROGRAM 


Table 1 

.094397 
=, 006346 =. 
=, 064816 =~, 
-.001169 
4385. 7347 
.0483 .0435 
.0168 .0152 
.0059 .0053 
0020 -. 0018 
.0007 .0006 
.0000 -.0000 
.0008 -.0015 
.0035 -.0017 
.0198 0189 
0397 <.0457 
.0291 .0500 
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055181 
002682 


045444 
003120 


wtian 
0397 


4.36 
.0048 


Aa 8 
.0006 


.0000 
-.0024 


.0006 
.0168 


-.0499 
~O712 


-059859 
-Ot2707 
.012473 
.003903 
.004679 
01355 
.001524 
.000497 


~061511 


000967 
002890 


.021904 
-.000174 


.007674 
.001807 


-1010 
Fi 8 Be D- 


0123 
.0043 


-QOTS 
.0005 


0002 .. 
,0034 +. 


«0033 
-O130 


0514 4 
0910 


.054098 
.006652 
.012680 
.001948 
.004723 
.000653 
0073515 
.000324 


«030351 
.003987 
.O0TST?7 
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=~, OOC7 1S 


.027651 
~ 000303 


«0909 
34 7 


.0110 
- 0039 


.0013 
.0005 


.0818 
~ 0265 


. 0099 
.0035 


.0012 
.0004 


.0005 
iat 


.0096 
.0014 


=,a>3 
«1208 


.047646 
eS sg Be 


12202 
.000148 


-004512 
.000022 


.001428 
~000179 


- 038992 


-0G35579 
.001247 
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Table 1 
(Continued) 
STAGE 2 
.182858  .160221 120688 .073819 .029920 -.003025 -.021307 
=<. D2S777 =. 020899 =~. 041198. =. 007a19 ~~ 607588 = . 005662. 004955 
003002 .001103 -.000076 -.000482 -.000407 -.000202 
STAGE 3 
.430137  .168113 -.068326 -.080180 .026939 .054430 -.010516 
-.041032 .001691 039772 003666 =-.0255T0. -. 007056 -;820730 
009159 -.015612 -.010347 .011750 .010849 -.008433 -.010828 
.005600 .010404 -.003210 -.009681 '@01235 -° .008742 ~*~ 2000352 
= 00766. = .001579)22 3,006 SNSec. 002478"\7- 700835. 4 003079.' . 004279 
003422. -.003148 -.003642  .002179 2003484 -.00133270;.0022R0 
.000614 .002976 -.000036 -.002601 .000410 .002192 #.000727 
-.001770 -.000931 001366 .001033 -.000990 -.001054 .000662 
001008 -.000381 .000918 .000160 .000794 .000009 -.000658 
=;900123 .000517 .- -800796. --.000s87 -=.000226- ...000273 .. .000255 
~.000193 -.000229 .000174 .000422 #.000276 
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Reviewers 


The successful completion of this program book represents the culmination of a joint effort on the part 
of many contributors. The purpose of this section is to acknowledge the extensive efforts expended by 
the reviewers. Their comprehensive testing contributed to the high quality of the programs included in 
the book. 


The following is a complete list of program titles, authors and their affiliations, reviewers and their 
affiliations, as well as computers, operating systems, and FORTRAN compilers used by the reviewers. 


1.1 FOUREA - A Short Demonstration Version of the FFT 
C. M. Rader 
M.I.T. Lincoln Laboratory, Lexington, MA 02173 
Reviewed by M. T. Dolan 
Bell Laboratories, Murray Hill, NJ 07974 
Honeywell 6080N computer 
GCOS 
FORTRAN Y compiler 


1.2 Fast Fourier Transform Algorithms 
G. D. Bergland and M. T. Dolan 
Bell Laboratories, Murray Hill, NJ 07974 
Reviewed by L. R. Morris 
Carleton University, Ottawa, Canada 
PDP-11/55, 11/60 computers 
RSX-11M 
FORTRAN IV PLUS compiler 
RT-11 
FORTRAN IV compiler 


1.3. FFT Subroutines for Sequences With Special Properties 
L. R. Rabiner 
Bell Laboratories, Murray Hill, NJ 07974 
Reviewed by M. T. Dolan 
Bell Laboratories, Murray Hill, NJ 07974 
Honeywell 6080N computer 
GCOS 
FORTRAN Y compiler 


1.4 Mixed Radix Fast Fourier Transforms 
R. C. Singleton 
SRI International, Menlo Park, CA 94025 
Reviewed by M. T. Dolan 
Bell Laboratories, Murray Hill, NJ 07974 
Honeywell 6080N computer 
GCOS 
FORTRAN Y compiler 
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1.5 Optimized Mass Storage FFT Program 
D. Fraser 
CSIRO, Canberra City, Australia 
Reviewed by N. M. Brenner 
IBM, Yorktown Heights, NY 10598 
IBM 370/168 computer 
VM/370 
FORTRAN IV compiler 
Reviewed by C. A. McGonegal, L. R. Rabiner, and M. T. Dolan 
Bell Laboratories, Murray Hill, NJ 07974 
Data General Eclipse S/230 computer 
RDOS 
FORTRAN V compiler 
Honeywell 6080N computer 
GCOS 
FORTRAN Y compiler 


1.6 Chirp z-Transform Algorithm Program 
L. R. Rabiner 
Bell Laboratories, Murray Hill, NJ 07974 
Reviewed by R. M. Mersereau 
Georgia Institute of Technology, Atlanta, GA 30332 
Data General Eclipse S/230 computer 
RDOS 
FORTRAN 5 compiler (Rev. 6) 


1.7 Complex General-N Winograd Fourier Transform Algorithm (WFTA) 
J. H. McClellan and H. Nawab 
M.I.T., Cambridge, MA 02139 
Reviewed by D. Kolba 
Rice University, Houston, TX 02881 
IBM 370/155 computer 
TSO 
FORTRAN GI compiler 


1.8 Time-Efficient Radix-4 Fast Fourier Transform 
L. R. Morris 
Carleton University, Ottawa, Canada 
Reviewed by C. M. Rader 
M.I.T. Lincoln Laboratory, Lexington, MA 02173 
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